-- Logs begin at Mon 2022-10-17 04:49:10 UTC, end at Mon 2022-10-17 06:21:19 UTC. -- Oct 17 05:20:30.254243 dv-cinder-861566-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 05:20:30.264485 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 05:20:30 2022] *** Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: nodename: dv-cinder-861566-1-iscsi Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: machine: x86_64 Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: clock source: unix Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: pcre jit disabled Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: detected number of CPU cores: 4 Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: current working directory: / Oct 17 05:20:30.266475 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: detected binary path: /usr/bin/uwsgi-core Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: your processes number limit is 47680 Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: your memory page size is 4096 bytes Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: detected max file descriptor number: 2048 Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: lock engine: pthread robust mutexes Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: thunder lock: enabled Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 05:20:30.267115 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 05:20:30.306491 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: Python main interpreter initialized at 0x561ffaa149c0 Oct 17 05:20:30.306491 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: python threads support enabled Oct 17 05:20:30.306491 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: your server socket listen backlog is limited to 100 connections Oct 17 05:20:30.306491 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: your mercy for graceful operations on workers is 90 seconds Oct 17 05:20:30.306809 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 05:20:30.306906 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: *** Operational MODE: preforking *** Oct 17 05:20:30.307171 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: *** uWSGI is running in multiple interpreter mode *** Oct 17 05:20:30.307171 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: spawned uWSGI master process (pid: 81570) Oct 17 05:20:30.307383 dv-cinder-861566-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 05:20:30.307933 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: spawned uWSGI worker 1 (pid: 81581, cores: 1) Oct 17 05:20:30.308233 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: spawned uWSGI worker 2 (pid: 81582, cores: 1) Oct 17 05:20:30.308384 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 05:20:34.614495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:20:34.614495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: from cryptography.utils import int_from_bytes Oct 17 05:20:34.615431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:20:34.615431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: from cryptography.utils import int_from_bytes Oct 17 05:20:34.615567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:20:34.615567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: from cryptography.utils import int_from_bytes Oct 17 05:20:34.615987 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:20:34.615987 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: from cryptography.utils import int_from_bytes Oct 17 05:20:34.672879 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 05:20:34.672879 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 05:20:34.673104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 05:20:34.673187 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] command line args: [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 05:20:34.673261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 05:20:34.673261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] command line args: [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 05:20:34.673373 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 05:20:34.673373 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ================================================================================ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 05:20:34.673516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 05:20:34.673516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ================================================================================ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 05:20:34.673640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] admin_token = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.673698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] admin_token = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.673760 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_dir = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.673760 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.673872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_dir = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.673937 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_source = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.673996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674058 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] debug = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_source = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674541 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] debug = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674541 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674683 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674744 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674868 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674933 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.674996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675175 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675244 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_config_append = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_config_append = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675700 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675814 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.675894 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_dir = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_dir = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676373 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676644 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676836 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.676916 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677045 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677128 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677316 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677512 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677595 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677790 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677864 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.677977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678062 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678139 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678334 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678526 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.678929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679007 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679241 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] publish_errors = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] publish_errors = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679647 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679791 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.679923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.680004 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.680347 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.680422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.680498 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] standard_threads = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681278 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] standard_threads = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681353 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681478 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681629 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681784 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681859 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_journal = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.681930 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_journal = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682004 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_json = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_json = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_stderr = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_stderr = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682305 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_syslog = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682381 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_syslog = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:20:34.682611 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.682684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.682684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.682873 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.682945 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683019 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683241 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683583 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683661 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683813 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683884 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.683958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684037 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684191 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684481 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684683 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684753 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684904 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.684975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685056 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685129 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685276 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685345 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685415 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685492 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685680 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685825 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685895 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.685972 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686050 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686121 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686273 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686541 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686701 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686786 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.686930 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687008 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687411 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687483 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687714 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.687970 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688132 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688209 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688360 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.688910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689077 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689313 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689558 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689788 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.689864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690021 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690254 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690338 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690666 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690873 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.690968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691335 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691456 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691829 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691917 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.691981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692047 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692165 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692754 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.692998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693310 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693769 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.693924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694619 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694695 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.694855 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695096 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695096 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695352 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695477 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695801 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.695801 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696199 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696442 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696566 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.696968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697041 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697335 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697406 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697731 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.697972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698120 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698202 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698353 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698599 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.698912 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.699072 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.699151 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.699329 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.699411 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.703769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.703881 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.703960 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704486 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704778 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.704995 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705067 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705367 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705437 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705592 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705665 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705927 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.705997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706076 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706297 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706377 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.706958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707031 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707241 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707345 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707433 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707508 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.707936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708014 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708187 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.external = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.external = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708334 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708730 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708803 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.password = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708874 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.password = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.708944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.token = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709023 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.token = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709238 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709309 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709380 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709809 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.709953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710095 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710165 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710165 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710303 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710460 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710687 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710761 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710832 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.710973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711043 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711120 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711199 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.711740 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712306 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712451 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712529 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712674 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712822 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712899 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.712971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713172 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713407 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713652 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713730 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.713975 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714200 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714446 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714681 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714851 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.714928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715364 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715669 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.715919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716068 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716785 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.716864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717015 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717485 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717564 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.717963 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718039 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718210 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718752 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718893 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.718995 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719383 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719626 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719879 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.719957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720121 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720466 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720625 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.720946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721090 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721402 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721786 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.721863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722265 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722510 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722587 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.722983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.723068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.723225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.723372 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.723520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.723595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.728885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.728982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729121 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729335 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729488 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729559 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729772 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729842 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.729986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730057 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730128 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730421 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730563 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730741 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.730954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731292 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731380 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731451 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731522 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731897 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.731968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732047 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732121 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732193 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732335 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732406 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732486 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732631 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732772 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732916 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.732990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733061 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733132 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733207 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733286 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733580 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733653 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733732 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.733949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734091 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734243 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734315 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734386 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734456 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734663 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734844 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734905 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.734974 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735125 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735266 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735370 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735815 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.735885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736030 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736234 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736305 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736375 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736781 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736859 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.736932 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737423 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.737980 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738148 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738227 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738710 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738871 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.738951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739842 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.739919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740071 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.740886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741134 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741304 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741646 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741812 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.741902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742038 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742117 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742345 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742596 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742760 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.742838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743276 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743764 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.743918 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744008 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744248 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744662 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744747 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744907 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.744985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745139 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745392 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745881 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.745959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746116 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746202 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746347 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.746909 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747341 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747819 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.747898 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748133 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748369 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748748 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748826 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.748981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749315 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749796 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.749956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750444 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750524 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750676 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750753 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.750990 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751689 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.751931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752014 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752255 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752423 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752681 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.752922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753007 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753245 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753478 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753879 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.753958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754196 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754352 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754578 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754819 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.754897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755141 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755361 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755444 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755550 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755629 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.755885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756027 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756496 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756582 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756812 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.756967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.757043 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:20:34.757136 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 05:20:34.757222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 05:20:34.786823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.787067 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.787158 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.787357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.787472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.787541 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.795773 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.795871 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.826530 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.826758 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.826849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.827005 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.885079 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.885562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.934240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.934415 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.938105 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:34.938339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092338 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092778 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.092998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.093091 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.093215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.093299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.093490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.093706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.116872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117250 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117341 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117486 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117797 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.117935 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.118017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.118208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.118301 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.118424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.118671 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.135993 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.136240 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.148551 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.148782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.148933 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149011 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149456 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149538 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149671 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149889 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.149980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.150104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.150318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:20:35.173953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.174159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:20:35.174471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.174562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.174685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:20:35.174769 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:20:35.174958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.175041 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.175209 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:20:35.175317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.175494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.175577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.175750 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.175844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.176008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.176266 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.185887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.186119 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.186391 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.186674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.186971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.187247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.187632 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.193662 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.194631 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.196606 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.196891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.197179 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.199893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.199988 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.200068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200144 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200220 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200386 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200465 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200541 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200852 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.200928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.201003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.201078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.201078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.201078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.201078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.221836 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.223195 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.223326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.223594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.224684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.224954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.228776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.229065 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.230206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.230288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.230488 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.230766 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.235685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.235976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.237193 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.237479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.241572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.241872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.242165 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.242459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.242739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.243063 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.243689 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.243857 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.243983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.244130 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.244272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.244558 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.244837 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.245110 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.245377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.245643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.259626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.259924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.261228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.261513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.263793 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.265396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.267015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.267333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.268692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.268987 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.271545 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.273312 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.274787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.276527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.278020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.278321 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.278608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.278897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.279182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.279495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.279759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.280060 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.280343 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.280621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.280902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.281177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.296057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.296350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.296619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.296888 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.298218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.298508 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.298778 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.299084 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.303759 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.305299 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.306036 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.306913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.307196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.307590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.309177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.309491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.310028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.310316 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.310601 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.310887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.311284 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.312286 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.312571 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.312858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.313147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.313567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.322976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.323259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.323570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.323858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.324241 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.324525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.324826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.325347 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.325634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.325917 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.326199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.326579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.326870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.327152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.340262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.342722 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.343156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.345678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.346047 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.347671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.348607 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.349295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.349590 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.349997 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.350093 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.351576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.351860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.352263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.356903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.358364 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.359206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.360713 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.361292 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.361694 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.361984 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.362264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.363617 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.364014 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.364300 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.364576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.370153 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.370448 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.370730 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.372621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.372919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.373203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.377621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.377907 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.378192 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.378473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.378750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.379025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.379324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.379617 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.379900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.380093 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:20:35.380178 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.380378 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.380459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.380660 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.380752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.380935 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.381017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.381208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.381300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.381485 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.381809 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.382104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.382369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.382669 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.382942 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.383218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.383507 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.384294 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:20:35.416339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561ffaa149c0 pid: 81581 (default app) Oct 17 05:20:35.419719 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee39b6-192f-4788-a474-7bd46c0ffa90 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:35.419821 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561ffaa149c0 pid: 81582 (default app) Oct 17 05:20:35.420009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee39b6-192f-4788-a474-7bd46c0ffa90 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:35.420285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee39b6-192f-4788-a474-7bd46c0ffa90 None None] PATH_INFO: `/v3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:35.424398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon Oct 17 05:20:35 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:42.953855 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:42.954379 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:42.954379 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:42.958664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:42.964628 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:42.965126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:42.965227 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:43.034690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: INFO dbcounter [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] Registered counter for database keystone Oct 17 05:20:43.054342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG oslo_db.sqlalchemy.engines [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81581) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 05:20:43.060527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writer thread running {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 05:20:43.237814 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG passlib.handlers.bcrypt [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81581) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 05:20:43.238118 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG passlib.handlers.bcrypt [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81581) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 05:20:43.295805 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:43.297398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:43.422971 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:20:43.931720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:44.003609 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: INFO dbcounter [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] Registered counter for database keystone Oct 17 05:20:44.025071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG oslo_db.sqlalchemy.engines [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81582) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 05:20:44.031196 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writer thread running {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 05:20:44.149227 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca19e26-ed37-407c-867a-c5b21759c927 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:44.153412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:44.154032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:44.154309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:44.154581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:44.162814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:44.213885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: WARNING keystone.common.rbac_enforcer.enforcer [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 05:20:44.214178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d2230-e17a-4c57-b6e2-ea04ebf1749e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:44.223170 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Oct 17 05:20:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 05:20:44.229598 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:44.246662 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b77d7-be53-4502-be40-d128f4208b5a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:44.251059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:44.252187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:44.252467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:44.252738 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] PATH_INFO: `/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:44.254303 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] RBAC: Authorizing `identity:get_project(project_id=29daf56ceab84ba1bd5c5b6f664b751b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:44.306782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 05:20:44.307126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f976e8c-ae64-4204-85ee-86717b58c3e7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:44.311597 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:20:44 2022] GET /identity/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:46.394204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098c30-71ce-4dd5-9e9e-d5c5140dd207 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:46.394747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098c30-71ce-4dd5-9e9e-d5c5140dd207 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:46.394747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098c30-71ce-4dd5-9e9e-d5c5140dd207 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:46.399585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:46.405087 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:46.405504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:46.405712 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:46.425591 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:46.427107 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:46.434876 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:20:46.703899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098c30-71ce-4dd5-9e9e-d5c5140dd207 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:46.715658 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098c30-71ce-4dd5-9e9e-d5c5140dd207 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:46.719801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a0f73-94b1-499d-bd1c-833b7ff7aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:46.720419 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a0f73-94b1-499d-bd1c-833b7ff7aa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:46.720695 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a0f73-94b1-499d-bd1c-833b7ff7aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:46.720964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a0f73-94b1-499d-bd1c-833b7ff7aa90 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:46.728394 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a0f73-94b1-499d-bd1c-833b7ff7aa90 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:46.730129 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a0f73-94b1-499d-bd1c-833b7ff7aa90 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:46.738519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Oct 17 05:20:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 05:20:46.744243 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:46.753111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19491f2b-1f0b-4940-8feb-ca28e0fd123e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:46.757453 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b469d-d88e-4316-9ae4-603b6c61b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:46.758071 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b469d-d88e-4316-9ae4-603b6c61b897 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:46.758910 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b469d-d88e-4316-9ae4-603b6c61b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:46.759182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b469d-d88e-4316-9ae4-603b6c61b897 admin admin] PATH_INFO: `/v3/users/d7f0e8d5f450484e921b5c820f033ee1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:46.760522 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b469d-d88e-4316-9ae4-603b6c61b897 admin admin] RBAC: Authorizing `identity:get_user(user_id=d7f0e8d5f450484e921b5c820f033ee1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:46.762333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b469d-d88e-4316-9ae4-603b6c61b897 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:46.763863 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon Oct 17 05:20:46 2022] GET /identity/v3/users/d7f0e8d5f450484e921b5c820f033ee1 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:48.912473 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74b911-e5db-447c-b85d-50d44b55c049 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:48.913050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74b911-e5db-447c-b85d-50d44b55c049 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:48.913050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74b911-e5db-447c-b85d-50d44b55c049 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:48.914815 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:48.919944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:48.920354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:48.920632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:48.938933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:48.940156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:48.992950 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d549c66-1753-4376-b7db-38ff338a9dbd None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:48.993234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d549c66-1753-4376-b7db-38ff338a9dbd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:48.993511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d549c66-1753-4376-b7db-38ff338a9dbd None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:48.994823 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:20:48.999907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.000186 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.000459 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.020878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:49.043217 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG passlib.handlers.bcrypt [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81582) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 05:20:49.043621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG passlib.handlers.bcrypt [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81582) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 05:20:49.093162 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:49.094711 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:49.191944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:20:49.287019 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.312003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:49.389921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453063-8bd4-422c-a7bf-731c476ad1dd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.424978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.425617 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.425893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.426163 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.437581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:20:49.446343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:20:49.447484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.448378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:20:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:20:49.453311 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.459657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.540143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:49.551199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62fe5-bbca-4cdf-829c-e2864e37c8e5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.553617 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:49.559569 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433ead9-5073-42ab-9116-4e5c810773b5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.588594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:20:49.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:49.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:49.593600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.594202 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.594478 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.594749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.595450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:49.596818 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:49.604186 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:20:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:20:49.609734 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.634810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.sql.core [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:20:49.634810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:20:49.634810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [parameters: {'id': 'de697cb571f64e63b283a7bbb0b5c1a0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:20:49.634810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:20:49.652593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:20:49.653414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application yield resource Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise self.value Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application session.commit() Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:20:49.654306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:20:49.655158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:20:49.656078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:20:49.656861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [parameters: {'id': 'de697cb571f64e63b283a7bbb0b5c1a0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:20:49.657657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:20:49.658379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.659225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 8/16] 10.222.0.13 () {60 vars in 1137 bytes} [Mon Oct 17 05:20:49 2022] POST /identity/v3/domains => generated 190 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:20:49.661762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.708265 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15e3c7-c7d2-4aae-8c0b-dee32c38de40 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.741591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.742955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.743238 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.743528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] PATH_INFO: `/v3/users/d7f0e8d5f450484e921b5c820f033ee1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.744979 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] RBAC: Authorizing `identity:get_user(user_id=d7f0e8d5f450484e921b5c820f033ee1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:49.746424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:49.748085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:20:49 2022] GET /identity/v3/users/d7f0e8d5f450484e921b5c820f033ee1 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:20:49.753557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.757790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3462ce-76ff-4866-9e04-7f225a267514 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.787422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:49.794616 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.795783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.795783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.795951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.806345 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:49.807852 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:49.814614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:20:49.823103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:20:49.828494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:20:49.829572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:20:49.829572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:20:49.829572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:20:49.829572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:20:49.830852 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.853559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309ea35-cc49-413c-9731-da79d66a3cdb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.885885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.886513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.886791 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.887069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.888198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:49.889645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:49.891173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:20:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:20:49.896779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:49.930317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa88a-270c-4e5d-82c4-fbc31917ddf5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.962666 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:49.963207 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:49.963519 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:49.963814 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:49.964457 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:20:49.964761 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:49.966131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba62f-3e49-4511-b070-5abe530aafa5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:49.983361 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:20:49.996794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249dc7-4caa-45c1-a189-79e3d7a400b0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:49.998532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:50.034692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e0a2-be85-4508-bb6f-fa0e9e4eeea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:50.035675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e0a2-be85-4508-bb6f-fa0e9e4eeea8 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:50.035957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e0a2-be85-4508-bb6f-fa0e9e4eeea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:50.036236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e0a2-be85-4508-bb6f-fa0e9e4eeea8 None admin] PATH_INFO: `/v3/domains/default/users/d7f0e8d5f450484e921b5c820f033ee1/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:50.038300 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e0a2-be85-4508-bb6f-fa0e9e4eeea8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d7f0e8d5f450484e921b5c820f033ee1, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:50.040424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e0a2-be85-4508-bb6f-fa0e9e4eeea8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:50.238237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 11/21] 10.222.0.13 () {58 vars in 1361 bytes} [Mon Oct 17 05:20:49 2022] PUT /identity/v3/domains/default/users/d7f0e8d5f450484e921b5c820f033ee1/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:20:52.760564 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e426a2-5756-445b-a7af-9276f73048e1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:52.761041 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e426a2-5756-445b-a7af-9276f73048e1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:52.761041 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e426a2-5756-445b-a7af-9276f73048e1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:52.762371 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:52.767492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cca083-e330-42a2-9933-945bc446b354 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:52.767632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cca083-e330-42a2-9933-945bc446b354 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:52.767912 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cca083-e330-42a2-9933-945bc446b354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:52.771209 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ab9de-c16f-49e5-b34f-75c7fe79aefc None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:52.771663 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ab9de-c16f-49e5-b34f-75c7fe79aefc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:52.771850 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ab9de-c16f-49e5-b34f-75c7fe79aefc None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:52.791340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 12/23] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:52 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:52.797824 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c2aca-1bf2-4d3d-a6ae-4b3c5cb99a6e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:52.798119 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c2aca-1bf2-4d3d-a6ae-4b3c5cb99a6e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:52.801404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c2aca-1bf2-4d3d-a6ae-4b3c5cb99a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:52.805106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d7cca083-e330-42a2-9933-945bc446b354 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:52.806350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d7cca083-e330-42a2-9933-945bc446b354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:52.835946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-580c2aca-1bf2-4d3d-a6ae-4b3c5cb99a6e None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:52.838184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-580c2aca-1bf2-4d3d-a6ae-4b3c5cb99a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:52.903226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:52.934005 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 12/24] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:52.953600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:52.960861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d3945-ec9a-4bae-8062-5e9d83b7b128 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:52.961717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d3945-ec9a-4bae-8062-5e9d83b7b128 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:52.961971 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d3945-ec9a-4bae-8062-5e9d83b7b128 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:52.963330 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:52.968810 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:52.969104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:52.969388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:53.000287 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:53.001487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:53.093525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 14/27] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:53.241223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d3945-ec9a-4bae-8062-5e9d83b7b128 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:53.328400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:53.360215 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:53.402127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d3945-ec9a-4bae-8062-5e9d83b7b128 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:53.450380 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e277743-49df-453c-8501-b579016cd1b1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:53.486111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18536e42-ba4d-4649-840a-5a6bf4efc6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:53.487109 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18536e42-ba4d-4649-840a-5a6bf4efc6f5 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:53.487570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18536e42-ba4d-4649-840a-5a6bf4efc6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:53.487978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18536e42-ba4d-4649-840a-5a6bf4efc6f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:53.488773 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18536e42-ba4d-4649-840a-5a6bf4efc6f5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:53.489531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2f5cc-ad97-4018-8d1d-d968416d5b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:53.493181 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2f5cc-ad97-4018-8d1d-d968416d5b23 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:53.493299 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18536e42-ba4d-4649-840a-5a6bf4efc6f5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:53.493387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2f5cc-ad97-4018-8d1d-d968416d5b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:53.493387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2f5cc-ad97-4018-8d1d-d968416d5b23 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:53.493387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2f5cc-ad97-4018-8d1d-d968416d5b23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:53.493901 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2f5cc-ad97-4018-8d1d-d968416d5b23 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:53.582276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 15/28] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:20:53 2022] POST /identity/v3/roles => generated 219 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:20:53.587325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8fa37-07f9-4cd3-858a-30ac1101cefb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:53.587325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8fa37-07f9-4cd3-858a-30ac1101cefb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:53.587325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8fa37-07f9-4cd3-858a-30ac1101cefb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:53.587325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 16/29] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:53.587325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8fa37-07f9-4cd3-858a-30ac1101cefb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:53.682726 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 14/30] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:20:53 2022] POST /identity/v3/roles => generated 225 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:20:53.684694 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:53.685179 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:53.685627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:53.707242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:53.709940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:53.776843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:53.803132 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:53.827831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 15/31] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:53.857210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8fa37-07f9-4cd3-858a-30ac1101cefb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:53.921108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:53.921777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:53.922061 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:53.922337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:53.954544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:53.955997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:20:53.979342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:20:53.982048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:20:53.982924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:20:53.982924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:20:53.982924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:20:53.982924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 17/32] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:20:53.987844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.130656 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3d37-497c-4e30-9116-0e5bc98b246c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.163376 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.181832 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:54.182826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.183394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.183888 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.184687 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:20:54.185171 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:54.186732 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a367518-9e44-4ab3-aec2-c4878c807a79 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:54.201565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 16/33] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:54.203037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cce3-555e-416f-a485-8d2e57bed26c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.203357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cce3-555e-416f-a485-8d2e57bed26c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.203641 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cce3-555e-416f-a485-8d2e57bed26c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.205259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 17/34] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:54.207558 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cce3-555e-416f-a485-8d2e57bed26c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.254667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:54.287655 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37e3b5-d8c9-4f70-b832-ed610bcc969f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.320298 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cce3-555e-416f-a485-8d2e57bed26c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.324052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9c249-e636-49e6-a7f2-0aaf1cb449fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:54.324865 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9c249-e636-49e6-a7f2-0aaf1cb449fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.325035 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9c249-e636-49e6-a7f2-0aaf1cb449fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.325314 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9c249-e636-49e6-a7f2-0aaf1cb449fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.327333 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:54.337865 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9c249-e636-49e6-a7f2-0aaf1cb449fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b1c9c249-e636-49e6-a7f2-0aaf1cb449fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:20:54.343954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:20:54.344779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:20:54.345606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 18/35] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:20:54.346364 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2b439-7c0c-4b40-85c4-5eb4ce41d70b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.346657 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2b439-7c0c-4b40-85c4-5eb4ce41d70b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.346943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2b439-7c0c-4b40-85c4-5eb4ce41d70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.358167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0ddcb-bd05-4e06-aeb7-ddefd8d255da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:54.358775 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ddcb-bd05-4e06-aeb7-ddefd8d255da None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.359057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ddcb-bd05-4e06-aeb7-ddefd8d255da None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.359353 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ddcb-bd05-4e06-aeb7-ddefd8d255da None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.359995 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0ddcb-bd05-4e06-aeb7-ddefd8d255da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:54.361470 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0ddcb-bd05-4e06-aeb7-ddefd8d255da None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:54.365651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c0e2b439-7c0c-4b40-85c4-5eb4ce41d70b None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:54.366840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c0e2b439-7c0c-4b40-85c4-5eb4ce41d70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:54.432730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 19/36] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:54.434052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5c960-4a7c-47a2-a6d6-613c75ef7d9f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.434347 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5c960-4a7c-47a2-a6d6-613c75ef7d9f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.434626 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5c960-4a7c-47a2-a6d6-613c75ef7d9f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.435958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 20/37] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:54.436711 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5c960-4a7c-47a2-a6d6-613c75ef7d9f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.443923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 18/38] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:20:54 2022] POST /identity/v3/projects => generated 301 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:20:54.453992 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.454987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.455598 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.484839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:54.486551 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:54.562211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5c960-4a7c-47a2-a6d6-613c75ef7d9f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.591352 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:54.601406 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 19/39] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:54.607564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:54.607564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.607564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.607564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.607910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:20:54.607910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:54.607910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:54.619686 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 21/40] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:54.625026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.648054 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:54.710332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.735403 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92e100-59da-41a7-a6e1-1f052c61c224 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.827483 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314742f5-c45f-4a0e-8ad7-7a4634ce0653 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.863764 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:54.864691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.865071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.865359 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.866460 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:54.868535 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:54.870122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 22/41] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:20:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:20:54.875641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.880513 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:54.881158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:54.881442 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:54.881717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] PATH_INFO: `/v3/users/d7f0e8d5f450484e921b5c820f033ee1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:54.882914 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d7f0e8d5f450484e921b5c820f033ee1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:54.884409 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:54.886027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 20/42] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:20:54 2022] GET /identity/v3/users/d7f0e8d5f450484e921b5c820f033ee1 => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:54.886770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:54.932993 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:54.973236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:54.977493 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c0a35-4c29-478d-884d-e121ba4dcf96 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:54.985384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b2ff2-6b86-4638-9e4f-27d0d42be4c0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:55.009749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:55.010362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:55.010646 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:55.010953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:55.011630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:55.013074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:55.018047 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:55.018643 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:55.018923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:55.019196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:55.019887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:55.021234 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:55.085655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 21/43] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:20:54 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:20:55.086503 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:55.098807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 23/44] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:20:54 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:20:55.210226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072e366-244d-49ef-80e3-13b7c844c676 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:55.249770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e42e7-9bdc-43cb-8fc4-e4013cbf6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:55.250343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e42e7-9bdc-43cb-8fc4-e4013cbf6962 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:55.250624 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e42e7-9bdc-43cb-8fc4-e4013cbf6962 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:55.250899 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e42e7-9bdc-43cb-8fc4-e4013cbf6962 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:55.252145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e42e7-9bdc-43cb-8fc4-e4013cbf6962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:55.253685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e42e7-9bdc-43cb-8fc4-e4013cbf6962 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:55.262687 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 22/45] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:55.262907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:55.368177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d7eae-6f38-41f4-9624-ee4cdeb75565 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:55.375427 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:55.400839 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:55.401448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:55.401756 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:55.402027 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:55.402768 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.domain.id=default, user.id=d7f0e8d5f450484e921b5c820f033ee1) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:20:55.403072 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:55.404619 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fbfeb-b3e8-4667-a183-f562c387e4b7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:55.415645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 24/46] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Oct 17 05:20:55 2022] GET /identity/v3/role_assignments?user.id=d7f0e8d5f450484e921b5c820f033ee1&scope.domain.id=default&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:57.276359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc835a-d3a0-411d-b9e1-3dc654b162e7 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:57.276897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc835a-d3a0-411d-b9e1-3dc654b162e7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:57.276897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc835a-d3a0-411d-b9e1-3dc654b162e7 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:57.278245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 23/47] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:20:57.283496 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:57.283919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:57.284052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:57.302700 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:20:57.303919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:20:57.367565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 25/48] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:20:57.639504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc835a-d3a0-411d-b9e1-3dc654b162e7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:57.657525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:57.738653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc835a-d3a0-411d-b9e1-3dc654b162e7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:57.771064 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558f138-7cf8-408f-862d-cb4074494dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:57.771712 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558f138-7cf8-408f-862d-cb4074494dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:57.771994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558f138-7cf8-408f-862d-cb4074494dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:57.772268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558f138-7cf8-408f-862d-cb4074494dd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:57.773353 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558f138-7cf8-408f-862d-cb4074494dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:57.774799 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558f138-7cf8-408f-862d-cb4074494dd1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:57.776390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 24/49] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:20:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:20:57.781902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:20:57.876262 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:20:57.878902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd73b0-5121-43a2-9a5a-6de4ff739701 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:20:57.912052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b864053-259d-4bd5-8c4f-ffc22aaa96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:20:57.912589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b864053-259d-4bd5-8c4f-ffc22aaa96da None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:20:57.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b864053-259d-4bd5-8c4f-ffc22aaa96da None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:20:57.913148 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b864053-259d-4bd5-8c4f-ffc22aaa96da None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:20:57.913789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b864053-259d-4bd5-8c4f-ffc22aaa96da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:20:57.915793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b864053-259d-4bd5-8c4f-ffc22aaa96da None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:20:57.977802 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 26/50] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Oct 17 05:20:57 2022] POST /identity/v3/projects => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:21:00.089386 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344c00-e1e9-4ef4-a089-f7e7e6491f4e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:00.089902 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344c00-e1e9-4ef4-a089-f7e7e6491f4e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:00.090009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344c00-e1e9-4ef4-a089-f7e7e6491f4e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:00.091270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 25/51] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:00.096349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:00.096745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:00.096901 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:00.115076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:00.116288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:00.180302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 27/52] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:00.445589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0344c00-e1e9-4ef4-a089-f7e7e6491f4e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:00.528552 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:00.544804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0344c00-e1e9-4ef4-a089-f7e7e6491f4e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:00.576518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123ccf0-7860-49ad-9ab2-5856f1addcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:00.577137 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123ccf0-7860-49ad-9ab2-5856f1addcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:00.577423 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123ccf0-7860-49ad-9ab2-5856f1addcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:00.577700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123ccf0-7860-49ad-9ab2-5856f1addcc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:00.578791 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123ccf0-7860-49ad-9ab2-5856f1addcc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:00.580278 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123ccf0-7860-49ad-9ab2-5856f1addcc2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:00.581809 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:00.587826 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:00.664513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:00.681365 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71c443-a553-4804-9d59-ef9f75827591 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:00.715499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc467-f3f1-4750-a05a-7d7f76971e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:00.716099 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc467-f3f1-4750-a05a-7d7f76971e16 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:00.716377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc467-f3f1-4750-a05a-7d7f76971e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:00.716648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc467-f3f1-4750-a05a-7d7f76971e16 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:00.717282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc467-f3f1-4750-a05a-7d7f76971e16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:00.718749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc467-f3f1-4750-a05a-7d7f76971e16 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:00.896172 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 28/54] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:00 2022] POST /identity/v3/users => generated 275 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:04.104386 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27661-18d2-45c3-853f-6b8f280748b0 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.105414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27661-18d2-45c3-853f-6b8f280748b0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.105898 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27661-18d2-45c3-853f-6b8f280748b0 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.107466 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 27/55] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:04.124932 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15284d-6233-4f86-a179-a969147fa0da None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.125856 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15284d-6233-4f86-a179-a969147fa0da None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.126328 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15284d-6233-4f86-a179-a969147fa0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.154653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de5bd0-b86a-42a0-a611-0c01a6d28f5b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.155288 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de5bd0-b86a-42a0-a611-0c01a6d28f5b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.155547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2c15284d-6233-4f86-a179-a969147fa0da None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:04.156028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de5bd0-b86a-42a0-a611-0c01a6d28f5b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.156752 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2c15284d-6233-4f86-a179-a969147fa0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:04.157765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 28/56] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:21:04.163394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af982439-caa3-4096-8210-42bbf8027086 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.163963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af982439-caa3-4096-8210-42bbf8027086 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.164450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af982439-caa3-4096-8210-42bbf8027086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.195320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-af982439-caa3-4096-8210-42bbf8027086 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:04.195320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-af982439-caa3-4096-8210-42bbf8027086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:04.261621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 29/57] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:04.322273 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 29/58] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:04.509733 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bec35-00da-4032-9757-a51a88467400 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.510048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bec35-00da-4032-9757-a51a88467400 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.510300 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bec35-00da-4032-9757-a51a88467400 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.511690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 30/59] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:04.517169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d03ee4-6643-4866-ad53-82806670df24 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.517469 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d03ee4-6643-4866-ad53-82806670df24 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.517753 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d03ee4-6643-4866-ad53-82806670df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.537242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bec35-00da-4032-9757-a51a88467400 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:04.540403 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b4d03ee4-6643-4866-ad53-82806670df24 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:04.541864 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b4d03ee4-6643-4866-ad53-82806670df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:04.621591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:04.634000 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 30/60] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:04.635816 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5e2c-7520-49d3-94be-84bef52c8440 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.636287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5e2c-7520-49d3-94be-84bef52c8440 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.636734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5e2c-7520-49d3-94be-84bef52c8440 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.638177 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 31/61] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:04.644188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.644676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.645199 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.664229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bec35-00da-4032-9757-a51a88467400 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:04.672695 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:04.674108 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:04.685151 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:04.701531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:04.702261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.702647 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.703014 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.715935 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:04.722008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:04.722870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:04.723696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 31/62] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:04.724351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:04.763184 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 32/63] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:04.764298 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:04.863871 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a090fe2-e5e0-4cc5-a4b2-1fd4377f2b3f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:04.916565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f582-816d-4aca-b21c-57d8625742bb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:04.921256 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:04.938912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:04.939823 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.940248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.940705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.962121 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:04.968804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:04.969684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:04.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:04.973908 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 32/64] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:21:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:04.975079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:04.982201 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:04.983081 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:04.983784 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:04.984263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:04.985345 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:04.985653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:04.987050 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:04.999882 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 33/65] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.001045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.004131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.083678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e58763-8b51-42b0-af33-8c3701a7e31b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.101730 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6bdc6-2b87-4b82-9a8a-ca5033649591 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.118488 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd61f3-7c58-4fe3-9e5c-809f1374b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.119115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd61f3-7c58-4fe3-9e5c-809f1374b10d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.119431 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd61f3-7c58-4fe3-9e5c-809f1374b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.119710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd61f3-7c58-4fe3-9e5c-809f1374b10d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.129766 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd61f3-7c58-4fe3-9e5c-809f1374b10d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-59cd61f3-7c58-4fe3-9e5c-809f1374b10d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:05.134597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:05.135514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:05.139108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:05.139756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.139756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.140198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 33/66] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:05.140342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.140342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.140468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d049e9-1434-42b2-b56b-d2d7423ad081 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.140533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:05.140596 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d049e9-1434-42b2-b56b-d2d7423ad081 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.140667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.140732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d049e9-1434-42b2-b56b-d2d7423ad081 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.140732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 34/67] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:05.140732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d049e9-1434-42b2-b56b-d2d7423ad081 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.141179 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.150114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 34/68] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.150826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.191212 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.214817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.241030 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d049e9-1434-42b2-b56b-d2d7423ad081 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.247922 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1153458-0cca-486c-bc0b-680a27ebf911 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.275445 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.276073 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.276348 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.276634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] PATH_INFO: `/v3/users/3db07d1d0f574b668fc4e337a196088f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.281690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bbc91-e676-4160-aec8-f0af89e6746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.282309 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bbc91-e676-4160-aec8-f0af89e6746c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.282663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bbc91-e676-4160-aec8-f0af89e6746c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.282968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bbc91-e676-4160-aec8-f0af89e6746c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.291215 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] RBAC: Authorizing `identity:get_user(user_id=3db07d1d0f574b668fc4e337a196088f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.292690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.293084 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bbc91-e676-4160-aec8-f0af89e6746c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:05.294269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/users/3db07d1d0f574b668fc4e337a196088f => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.295029 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b50bbc91-e676-4160-aec8-f0af89e6746c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:05.298358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:05.299239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:05.300148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:05.304002 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:05.305191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.305491 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.305763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.323150 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:05.324385 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:05.389651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 36/71] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:05.390467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.393781 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a9b8c-35b2-4e29-a72b-91ce869beba2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.402549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.434974 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.435901 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.436017 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.436368 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.437029 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:05.437331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.438740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.447608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.448692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.457150 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.500408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b185c1-ef8f-4a4a-82c4-7239e3cd3139 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.533096 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.533718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.533999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.534272 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] PATH_INFO: `/v3/users/d7f0e8d5f450484e921b5c820f033ee1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.535663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] RBAC: Authorizing `identity:get_user(user_id=d7f0e8d5f450484e921b5c820f033ee1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.537223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.543318 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 37/73] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/users/d7f0e8d5f450484e921b5c820f033ee1 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.543318 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.556236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6356b-7e25-485b-af6f-c5cd667c7366 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.592017 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.592648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.592934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.593214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.594335 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.595802 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.597405 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 37/74] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.598158 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.639023 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.649106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4919c0-e038-449c-bace-a31f2e1266a9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.672559 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.683583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.684207 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.684496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.684772 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.685430 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:05.685738 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.687134 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.696192 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 38/75] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.696900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.705716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670102ce-8ce5-4414-bc00-0cf74f2a07e2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.738992 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.739651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.739934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.740210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] PATH_INFO: `/v3/users/3db07d1d0f574b668fc4e337a196088f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.741396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] RBAC: Authorizing `identity:get_user(user_id=3db07d1d0f574b668fc4e337a196088f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.743425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.745002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 38/76] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/users/3db07d1d0f574b668fc4e337a196088f => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.745002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.799914 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3464957-ccbd-4ff6-9379-bafdbcbad80f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.832739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.833359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.833639 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.833910 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.835019 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.836523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.838303 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 39/77] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.838874 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.846462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45154c53-7e90-45b4-9a12-f6966d8cfaea None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.848785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.880947 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.881597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.881881 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.882159 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de/users/3db07d1d0f574b668fc4e337a196088f/roles/d9150fbefb504f7d95662994e118df63` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.884096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] RBAC: Authorizing `identity:create_grant(project_id=17423390f676473593042b205bc4d4de, user_id=3db07d1d0f574b668fc4e337a196088f, role_id=d9150fbefb504f7d95662994e118df63)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.885523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.893255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:05.931043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 39/78] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:05 2022] PUT /identity/v3/projects/17423390f676473593042b205bc4d4de/users/3db07d1d0f574b668fc4e337a196088f/roles/d9150fbefb504f7d95662994e118df63 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:05.932067 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:05.944747 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833efbe-3f4a-47b5-8512-62f1fd4c0434 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:05.982141 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:05.982703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:05.982987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:05.983282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:05.984496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:05.985997 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:05.987684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 40/79] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:05.988493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:06.055926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd216be-a6bc-46d0-9950-6a5e485271fd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.095247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.095911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.096195 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.096465 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] PATH_INFO: `/v3/users/3db07d1d0f574b668fc4e337a196088f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.097614 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] RBAC: Authorizing `identity:get_user(user_id=3db07d1d0f574b668fc4e337a196088f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.099052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.100642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 40/80] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/users/3db07d1d0f574b668fc4e337a196088f => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:06.102742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:06.103485 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fbfd1-04c5-4641-bf95-d097ef5997c3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.132900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:06.136424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:06.144323 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.145470 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.145944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.146354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.151169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.151169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.151169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 41/81] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:05 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:06.151522 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:06.211182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ed2f-859d-4fd0-80b5-2a3b07710e6c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.246424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.247006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.247286 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.247589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de/users/d7f0e8d5f450484e921b5c820f033ee1/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.249484 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] RBAC: Authorizing `identity:create_grant(project_id=17423390f676473593042b205bc4d4de, user_id=d7f0e8d5f450484e921b5c820f033ee1, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.250915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.256548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c347509-ce6b-445c-9ea3-b69a139d6b5a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.291776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.292396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.292683 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.292960 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.293609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.295152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.330390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 41/82] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:06 2022] PUT /identity/v3/projects/17423390f676473593042b205bc4d4de/users/d7f0e8d5f450484e921b5c820f033ee1/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:06.331467 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:06.407339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 42/83] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:06 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:06.407339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:06.479342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:06.499949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4b46-42bd-4ed9-bbe8-527fd577fa97 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.530896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:06.566617 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.567386 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.567680 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.567961 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] PATH_INFO: `/v3/projects/02d9ad431af34eabbf202b8f45f31dcb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.570533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] RBAC: Authorizing `identity:get_project(project_id=02d9ad431af34eabbf202b8f45f31dcb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.572460 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.575917 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 42/84] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:06 2022] GET /identity/v3/projects/02d9ad431af34eabbf202b8f45f31dcb => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:06.582572 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:06.597768 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4f04f-c190-4ea7-977e-af134c852cce None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.650280 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d383e-25c7-4045-ad16-b9039e24ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.650963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d383e-25c7-4045-ad16-b9039e24ef6a None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.651250 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d383e-25c7-4045-ad16-b9039e24ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.651561 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d383e-25c7-4045-ad16-b9039e24ef6a None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de/users/3db07d1d0f574b668fc4e337a196088f/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.653780 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d383e-25c7-4045-ad16-b9039e24ef6a None admin] RBAC: Authorizing `identity:create_grant(project_id=17423390f676473593042b205bc4d4de, user_id=3db07d1d0f574b668fc4e337a196088f, role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.655266 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d383e-25c7-4045-ad16-b9039e24ef6a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.716106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7af3b2-ae19-4a26-bf5b-8b290fb38867 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:06.725911 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 43/85] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:06 2022] PUT /identity/v3/projects/17423390f676473593042b205bc4d4de/users/3db07d1d0f574b668fc4e337a196088f/roles/517b2b1aaac34e92b002aec2385431d0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:06.760942 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877300c-cc08-411d-af73-174d7f0b7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:06.761534 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877300c-cc08-411d-af73-174d7f0b7ab1 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:06.761813 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877300c-cc08-411d-af73-174d7f0b7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:06.762084 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877300c-cc08-411d-af73-174d7f0b7ab1 None admin] PATH_INFO: `/v3/projects/02d9ad431af34eabbf202b8f45f31dcb/users/3db07d1d0f574b668fc4e337a196088f/roles/d9150fbefb504f7d95662994e118df63` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:06.764252 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877300c-cc08-411d-af73-174d7f0b7ab1 None admin] RBAC: Authorizing `identity:create_grant(project_id=02d9ad431af34eabbf202b8f45f31dcb, user_id=3db07d1d0f574b668fc4e337a196088f, role_id=d9150fbefb504f7d95662994e118df63)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:06.765740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877300c-cc08-411d-af73-174d7f0b7ab1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:06.887389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 43/86] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:06 2022] PUT /identity/v3/projects/02d9ad431af34eabbf202b8f45f31dcb/users/3db07d1d0f574b668fc4e337a196088f/roles/d9150fbefb504f7d95662994e118df63 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:21:08.893354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a252e-085f-470c-b76e-f3850c035d1a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:08.894359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a252e-085f-470c-b76e-f3850c035d1a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:08.894854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a252e-085f-470c-b76e-f3850c035d1a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:08.896703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 44/87] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:08.902603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bff521-cb7a-4ddd-b8b1-d37545c22d79 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:08.903011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bff521-cb7a-4ddd-b8b1-d37545c22d79 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:08.903145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bff521-cb7a-4ddd-b8b1-d37545c22d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:08.932284 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c2bff521-cb7a-4ddd-b8b1-d37545c22d79 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:08.933701 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c2bff521-cb7a-4ddd-b8b1-d37545c22d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:08.949399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:09.017550 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 44/88] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:09.037131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796221d-b38c-417e-a274-b8185c4bea41 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.037754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796221d-b38c-417e-a274-b8185c4bea41 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.038245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796221d-b38c-417e-a274-b8185c4bea41 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.040368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 45/89] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:09.055079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235bee2-6024-4c7c-9621-a6385570f9f0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.055390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235bee2-6024-4c7c-9621-a6385570f9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.055672 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235bee2-6024-4c7c-9621-a6385570f9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.077382 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9235bee2-6024-4c7c-9621-a6385570f9f0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:09.078562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9235bee2-6024-4c7c-9621-a6385570f9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:09.162068 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 45/90] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:09.386074 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24631-a701-4a4a-9722-66c262efee16 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.386749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24631-a701-4a4a-9722-66c262efee16 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.387263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24631-a701-4a4a-9722-66c262efee16 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.388831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 46/91] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:09.394009 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.394299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.394575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.424370 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:09.425521 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:09.506060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24631-a701-4a4a-9722-66c262efee16 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:09.506616 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 46/92] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:09.697869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:09.715168 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:09.731698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24631-a701-4a4a-9722-66c262efee16 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:09.745924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:09.780391 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb744fd-83b6-49d2-a60e-27125688a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:09.781033 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb744fd-83b6-49d2-a60e-27125688a9ad None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.781313 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb744fd-83b6-49d2-a60e-27125688a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.781584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb744fd-83b6-49d2-a60e-27125688a9ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.793114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb744fd-83b6-49d2-a60e-27125688a9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4fb744fd-83b6-49d2-a60e-27125688a9ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:09.798027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:09.798886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:09.799678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:09.800270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 47/93] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:09.801546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dd941-2f97-4f72-bed1-ccf3a2038c0f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.801834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dd941-2f97-4f72-bed1-ccf3a2038c0f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.802112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dd941-2f97-4f72-bed1-ccf3a2038c0f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.803687 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 48/94] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:09.808148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dd941-2f97-4f72-bed1-ccf3a2038c0f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:09.917787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe36bcb-e7c1-490f-915e-af816a10ccf3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:09.941009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dd941-2f97-4f72-bed1-ccf3a2038c0f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:09.958677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:09.960151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.961247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.961676 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.974625 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:09.979106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:09.979939 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.980386 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.980799 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:09.981371 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:09.982495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:09.983269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 47/95] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:21:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:09.984313 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:09.985208 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:09.985726 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:09.987432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adadff-c7a6-4bb4-862e-cfaae6af53fe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:09.997222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 49/96] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:09.998776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:09.999273 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:09.999761 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.018329 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:10.019837 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:10.045366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.065909 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.096279 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 50/97] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:10.097491 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.123326 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72416c-19b6-494f-8002-d1e6dc1b6703 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.156165 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.156811 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.157208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.157461 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.169409 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:10.174718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:10.175582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:10.176362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 48/98] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:10.177006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.253988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b219d2-dc00-47ff-aa81-a9f6f7e1d636 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.290281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.291226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.291748 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.292213 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.293063 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:10.293604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.295182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.301045 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4347-2ace-4b14-9d78-d0a2849ca75d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.304126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 51/99] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:10.305198 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.328583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.337708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.345083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faaf6b-8788-4283-8c8e-30655cdc7d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.345598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaf6b-8788-4283-8c8e-30655cdc7d9a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.345880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaf6b-8788-4283-8c8e-30655cdc7d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.346160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaf6b-8788-4283-8c8e-30655cdc7d9a None admin] PATH_INFO: `/v3/users/3db07d1d0f574b668fc4e337a196088f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.347559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faaf6b-8788-4283-8c8e-30655cdc7d9a None admin] RBAC: Authorizing `identity:get_user(user_id=3db07d1d0f574b668fc4e337a196088f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.349125 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faaf6b-8788-4283-8c8e-30655cdc7d9a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.350803 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 49/100] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/users/3db07d1d0f574b668fc4e337a196088f => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:10.352201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2783e-4f20-41a6-b6fa-006d0748026b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.352427 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2783e-4f20-41a6-b6fa-006d0748026b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.352649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2783e-4f20-41a6-b6fa-006d0748026b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.354014 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 50/101] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:10.354870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2783e-4f20-41a6-b6fa-006d0748026b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.429770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad7c66-ef44-4cd6-bd02-262c0fbb9bfe None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.470430 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.470848 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2783e-4f20-41a6-b6fa-006d0748026b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.471029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.471330 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.471604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.472261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:10.472561 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.473959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fa645-7370-4940-b2a8-d50a5bb8d2b6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.483349 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 52/102] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:10.483472 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fd597-b575-4c78-838b-56601b1583ec None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.483826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fd597-b575-4c78-838b-56601b1583ec None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.484081 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fd597-b575-4c78-838b-56601b1583ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.501394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-484fd597-b575-4c78-838b-56601b1583ec None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:10.502572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-484fd597-b575-4c78-838b-56601b1583ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:10.504273 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.504873 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.505152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.505430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] PATH_INFO: `/v3/users/d7f0e8d5f450484e921b5c820f033ee1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.506525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] RBAC: Authorizing `identity:get_user(user_id=d7f0e8d5f450484e921b5c820f033ee1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.508029 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.509574 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 51/103] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/users/d7f0e8d5f450484e921b5c820f033ee1 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:10.510323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.572431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 53/104] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:10.573271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fd597-b575-4c78-838b-56601b1583ec None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.579840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.587161 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.616967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153577-6e41-4c92-8870-f904575c77f7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.652373 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.652982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.653269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.653550 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.672893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fd597-b575-4c78-838b-56601b1583ec None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.706647 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bddf-9853-40e9-88bb-f7f17c6d21ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.707282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bddf-9853-40e9-88bb-f7f17c6d21ce None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.707596 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bddf-9853-40e9-88bb-f7f17c6d21ce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.707876 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bddf-9853-40e9-88bb-f7f17c6d21ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.709379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1bddf-9853-40e9-88bb-f7f17c6d21ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.711410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1bddf-9853-40e9-88bb-f7f17c6d21ce None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.713138 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 54/105] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:10.714088 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10be6-5ae1-4c15-9e3f-50c59f8e7573 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.714374 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10be6-5ae1-4c15-9e3f-50c59f8e7573 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.714657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10be6-5ae1-4c15-9e3f-50c59f8e7573 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.719352 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 55/106] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:10.720711 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10be6-5ae1-4c15-9e3f-50c59f8e7573 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.729000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.730464 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.795853 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.807084 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:10.828351 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10be6-5ae1-4c15-9e3f-50c59f8e7573 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.837222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 52/107] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:10.837222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.863737 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.864466 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.864751 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.865026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] PATH_INFO: `/v3/users/0483b8003d9443aa8c71e7e958c62202` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.878461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] RBAC: Authorizing `identity:get_user(user_id=0483b8003d9443aa8c71e7e958c62202)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:10.879944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:10.886605 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 56/108] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/users/0483b8003d9443aa8c71e7e958c62202 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:10.886605 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:10.933123 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cd5f6-407a-40fc-be1c-ad371a74a17e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:10.966999 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:10.967632 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:10.967917 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:10.968190 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:10.982107 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5204f9-9a6f-4b17-bdff-459745d93911 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.000312 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.015573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cb1a8-a58e-429b-b493-a6dff1328637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.016179 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cb1a8-a58e-429b-b493-a6dff1328637 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.016468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cb1a8-a58e-429b-b493-a6dff1328637 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.016749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cb1a8-a58e-429b-b493-a6dff1328637 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.017394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cb1a8-a58e-429b-b493-a6dff1328637 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.018848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cb1a8-a58e-429b-b493-a6dff1328637 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.033068 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.034977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.085648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 57/109] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:21:10 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:11.088032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.088528 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.088978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.111440 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:11.112698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:11.146056 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 53/110] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:11.146056 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.188814 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 58/111] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:11.190021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.235738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.278831 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d2d4f-ddb6-43a0-9b75-1fdaadbd775a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.321565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.322208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.322494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.322770 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.323920 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.324947 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b641d-15d7-41c3-b816-edb9d57ecab3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.325443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.330309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 54/112] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:11.331127 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.357082 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.371762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.372488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.372836 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.373181 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.384202 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:11.389408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:11.390290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:11.391090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:11.395583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 59/113] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:11.396968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.463633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f4e4b-69b6-4e6e-b18f-14346d8236f1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.499822 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.500729 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.501006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.501291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.502414 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.503908 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.505541 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 55/114] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:11.506301 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.509503 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.514939 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913f41-81a0-47c4-984e-569bc48c0012 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.550118 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.550752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.551035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.551326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.552452 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.553903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.556072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 60/115] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:11.556834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.606341 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.613994 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4a7a-439d-451e-b748-f0e3a8b2fb5a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.649205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.649512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.649755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.650034 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.650755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] RBAC: Adding query filter params (role.id=d9150fbefb504f7d95662994e118df63, scope.project.id=17423390f676473593042b205bc4d4de, user.id=3db07d1d0f574b668fc4e337a196088f) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:11.651074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.652545 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.659026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 56/116] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/role_assignments?user.id=3db07d1d0f574b668fc4e337a196088f&scope.project.id=17423390f676473593042b205bc4d4de&role.id=d9150fbefb504f7d95662994e118df63 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:11.659921 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.671427 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cfe9d-7fb2-43f9-acfd-879293f74541 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.719804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.720994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.721418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.722095 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.723506 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:11.724145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.726240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.737845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 61/117] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:11.739557 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.747559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.793305 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68144732-9ee3-4250-a9d7-7f6f8dda48a0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.834803 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.835615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.835905 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.836296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.849117 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:11.855088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:11.856211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:11.857137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:11.857823 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 57/118] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:11.863515 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:11.882031 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dd0ed-9dea-4eae-b5a7-70a901dad1ae None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:11.899794 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:11.932282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:11.933083 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:11.933370 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:11.933658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de/users/0483b8003d9443aa8c71e7e958c62202/roles/319c0aa665e04428bed4854c6664943a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:11.935684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] RBAC: Authorizing `identity:create_grant(project_id=17423390f676473593042b205bc4d4de, user_id=0483b8003d9443aa8c71e7e958c62202, role_id=319c0aa665e04428bed4854c6664943a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:11.937726 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:11.981298 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e09c50-c195-4fd3-a552-50633c30ecd5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.005593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 62/119] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:11 2022] PUT /identity/v3/projects/17423390f676473593042b205bc4d4de/users/0483b8003d9443aa8c71e7e958c62202/roles/319c0aa665e04428bed4854c6664943a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:12.006629 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.010637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.025825 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.026495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.026785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.027121 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.028024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=17423390f676473593042b205bc4d4de, user.id=d7f0e8d5f450484e921b5c820f033ee1) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:12.028406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.032594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.038094 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 58/120] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:11 2022] GET /identity/v3/role_assignments?user.id=d7f0e8d5f450484e921b5c820f033ee1&scope.project.id=17423390f676473593042b205bc4d4de&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:12.039551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.147438 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d29c0-e3ef-4f7c-bfa5-9ef0989c9605 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.203239 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb4e49-b730-4ae3-97b9-77bb7050b186 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.230625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.231287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.231594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.231874 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] PATH_INFO: `/v3/users/3db07d1d0f574b668fc4e337a196088f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.233233 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] RBAC: Authorizing `identity:get_user(user_id=3db07d1d0f574b668fc4e337a196088f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.234725 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.237672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 63/121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/users/3db07d1d0f574b668fc4e337a196088f => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:12.243739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.273630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.274251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.274531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.274801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.275554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:12.275797 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.277258 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.287247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 59/122] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:12.295986 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.300883 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.369650 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.376727 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a857f-0112-4c10-b977-78845a79f240 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.428147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.429755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.430337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.431088 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.442540 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d99e-95db-40e1-84f6-34be5acdd657 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.490233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.491338 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.491673 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.491954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] PATH_INFO: `/v3/users/3db07d1d0f574b668fc4e337a196088f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.495435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=3db07d1d0f574b668fc4e337a196088f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.497261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.499770 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 60/123] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/users/3db07d1d0f574b668fc4e337a196088f => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:12.510805 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.532909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.534841 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.581915 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.637457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8de3-8466-43db-b134-173cfed42bc0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.645056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 64/124] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 05:21:12.651230 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.662319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.685016 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.686221 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.687079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.687834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.766413 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.768056 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.779928 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506a508-5277-4a9f-9ebc-ac0b8847b163 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.827803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:12.828926 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:12.829616 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:12.830055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:12.832043 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:12.834467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:12.837040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:12.843094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.855667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.881464 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 61/126] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:12.887778 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:12.916178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:12.962576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0eddf-3a8e-4355-8aa2-9c58b1349ca2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:12.999672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:13.000513 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:13.000893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:13.001262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:13.002182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] RBAC: Adding query filter params (role.id=517b2b1aaac34e92b002aec2385431d0, scope.project.id=17423390f676473593042b205bc4d4de, user.id=3db07d1d0f574b668fc4e337a196088f) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:13.002584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:13.004365 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:13.013412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 66/127] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/role_assignments?user.id=3db07d1d0f574b668fc4e337a196088f&scope.project.id=17423390f676473593042b205bc4d4de&role.id=517b2b1aaac34e92b002aec2385431d0 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:13.020189 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63967e69-75c5-4f1e-932e-fcbee6b2c71e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:13.062169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98e8d1-f485-4f62-9829-19e7b8915060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:13.063059 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98e8d1-f485-4f62-9829-19e7b8915060 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:13.063540 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98e8d1-f485-4f62-9829-19e7b8915060 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:13.063972 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98e8d1-f485-4f62-9829-19e7b8915060 None admin] PATH_INFO: `/v3/projects/02d9ad431af34eabbf202b8f45f31dcb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:13.065280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98e8d1-f485-4f62-9829-19e7b8915060 None admin] RBAC: Authorizing `identity:get_project(project_id=02d9ad431af34eabbf202b8f45f31dcb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:13.066882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98e8d1-f485-4f62-9829-19e7b8915060 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:13.068806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 62/128] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:12 2022] GET /identity/v3/projects/02d9ad431af34eabbf202b8f45f31dcb => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:13.078368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:13.156554 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:13.194689 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8708bc-ab97-4f43-91e5-c3d91a0f8d4a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:13.232195 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:13.233145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:13.233640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:13.234102 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:13.235009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] RBAC: Adding query filter params (role.id=d9150fbefb504f7d95662994e118df63, scope.project.id=02d9ad431af34eabbf202b8f45f31dcb, user.id=3db07d1d0f574b668fc4e337a196088f) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:13.235638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:13.237301 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20fd5-8b21-4434-bd8a-e2e832b37d2e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:13.244523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 67/129] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:13 2022] GET /identity/v3/role_assignments?user.id=3db07d1d0f574b668fc4e337a196088f&scope.project.id=02d9ad431af34eabbf202b8f45f31dcb&role.id=d9150fbefb504f7d95662994e118df63 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:13.402436 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec7a23-ccc1-4ac3-a40d-09ba0e0d6284 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:13.402754 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec7a23-ccc1-4ac3-a40d-09ba0e0d6284 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:13.402997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec7a23-ccc1-4ac3-a40d-09ba0e0d6284 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:13.404549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 63/130] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:13.412718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:13.412718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:13.412718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:13.431097 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:13.432332 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:13.501712 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 68/131] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:13.765535 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec7a23-ccc1-4ac3-a40d-09ba0e0d6284 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:13.851958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:13.864080 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec7a23-ccc1-4ac3-a40d-09ba0e0d6284 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:13.896564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151abdb9-b88c-4018-a964-011987e3b080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:13.897195 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151abdb9-b88c-4018-a964-011987e3b080 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:13.897475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151abdb9-b88c-4018-a964-011987e3b080 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:13.898277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151abdb9-b88c-4018-a964-011987e3b080 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:13.899400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151abdb9-b88c-4018-a964-011987e3b080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:13.900842 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151abdb9-b88c-4018-a964-011987e3b080 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:13.902382 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 64/132] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:13.907934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:13.967501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:14.004571 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fece6a-cee4-45f3-9b94-b4c9812ac6ce None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:14.036868 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3baabd-e920-4487-a6e1-5c70ceeea67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:14.037466 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3baabd-e920-4487-a6e1-5c70ceeea67f None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:14.037679 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3baabd-e920-4487-a6e1-5c70ceeea67f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:14.037952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3baabd-e920-4487-a6e1-5c70ceeea67f None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:14.039222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3baabd-e920-4487-a6e1-5c70ceeea67f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:14.040685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3baabd-e920-4487-a6e1-5c70ceeea67f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:14.113895 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 69/133] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:13 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:14.342169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40c7e4-d664-49b6-97c1-3ab8c2158f6a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:14.342664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40c7e4-d664-49b6-97c1-3ab8c2158f6a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:14.342747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40c7e4-d664-49b6-97c1-3ab8c2158f6a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:14.344174 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 65/134] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:14.349547 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:14.349834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:14.350112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:14.368839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:14.370038 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:14.440607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 70/135] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:14.709356 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40c7e4-d664-49b6-97c1-3ab8c2158f6a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:14.815237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40c7e4-d664-49b6-97c1-3ab8c2158f6a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:14.850206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:14.851781 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:14.852289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:14.852755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:14.863567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:14.869276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:14.870140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:14.870912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:14.873478 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 66/136] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:14.878961 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:14.931486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:14.982459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ae32-432b-4ab3-8314-528338bdd318 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:15.017889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:15.018860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:15.019391 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:15.019862 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:15.020716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:15.021264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:15.022868 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:15.031633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 71/137] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:15.040410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:15.060867 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:15.149757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121154e-51f3-416f-8aa1-b4329e83a242 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:15.186425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:15.187403 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:15.187913 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:15.188384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] PATH_INFO: `/v3/users/0483b8003d9443aa8c71e7e958c62202` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:15.189707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] RBAC: Authorizing `identity:get_user(user_id=0483b8003d9443aa8c71e7e958c62202)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:15.191437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:15.193535 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 67/138] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:15 2022] GET /identity/v3/users/0483b8003d9443aa8c71e7e958c62202 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:15.199388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:15.307219 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76487-4857-4171-b638-930b6c88c4ab None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:15.320661 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:15.345299 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:15.346326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:15.346833 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:15.347372 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:15.426151 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:15.428005 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:15.534536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 72/139] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 05:21:15.540656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:15.626363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:15.643096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66876576-67fd-4a78-8fc9-e4c84dc92c24 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:15.684084 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fa181-68cf-442f-af1d-880f1176cbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:15.684719 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fa181-68cf-442f-af1d-880f1176cbca None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:15.685003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fa181-68cf-442f-af1d-880f1176cbca None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:15.685281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fa181-68cf-442f-af1d-880f1176cbca None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:15.686586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fa181-68cf-442f-af1d-880f1176cbca None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:15.688193 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fa181-68cf-442f-af1d-880f1176cbca None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:15.690160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 68/140] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:15 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:15.696749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:15.711923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:15.807694 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25b8f8-d046-4272-b28d-a7a18f27952a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:15.851187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:15.851187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:15.851187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:15.851187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:15.851595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] RBAC: Adding query filter params (role.id=319c0aa665e04428bed4854c6664943a, scope.project.id=17423390f676473593042b205bc4d4de, user.id=0483b8003d9443aa8c71e7e958c62202) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:15.851595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:15.851595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fb9a4-2db2-469e-a1b7-fb241642bf71 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:15.855884 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 73/141] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:15 2022] GET /identity/v3/role_assignments?user.id=0483b8003d9443aa8c71e7e958c62202&scope.project.id=17423390f676473593042b205bc4d4de&role.id=319c0aa665e04428bed4854c6664943a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:16.397779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883baa-40c3-495c-a205-d337ae9a76fa None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.398063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883baa-40c3-495c-a205-d337ae9a76fa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.398490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883baa-40c3-495c-a205-d337ae9a76fa None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.399991 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 69/142] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:16.405096 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76c875-666e-42d2-a1a8-286c0c90ebbe None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.405921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76c875-666e-42d2-a1a8-286c0c90ebbe None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.406409 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76c875-666e-42d2-a1a8-286c0c90ebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.426202 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8f76c875-666e-42d2-a1a8-286c0c90ebbe None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:16.427672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8f76c875-666e-42d2-a1a8-286c0c90ebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:16.499654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 74/143] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:16.556822 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ee2f5-649a-48c7-9030-1d326b40740e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.557340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ee2f5-649a-48c7-9030-1d326b40740e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.557411 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ee2f5-649a-48c7-9030-1d326b40740e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.558753 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 70/144] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:16.564699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb9c92-ceb4-46ed-96e3-b8f671d50e54 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.565025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb9c92-ceb4-46ed-96e3-b8f671d50e54 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.565261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb9c92-ceb4-46ed-96e3-b8f671d50e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.578887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:16.585464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b8cb9c92-ceb4-46ed-96e3-b8f671d50e54 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:16.586655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8cb9c92-ceb4-46ed-96e3-b8f671d50e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:16.652465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 75/145] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:16.674652 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e597387-4f41-496b-83f3-5b8a826339fd None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.674944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e597387-4f41-496b-83f3-5b8a826339fd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.675225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e597387-4f41-496b-83f3-5b8a826339fd None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.676710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 71/146] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:16.682301 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.682593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.682867 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.700488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:16.701666 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:16.769542 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 76/147] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:16.775393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e597387-4f41-496b-83f3-5b8a826339fd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:16.874103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e597387-4f41-496b-83f3-5b8a826339fd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:16.907058 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:16.907651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:16.907931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:16.908200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:16.918876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:16.923745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:16.924799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:16.925638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 72/148] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:21:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:16.928272 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:16.930530 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:16.945129 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:16.997526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.032441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8654f90-81df-4db5-8a95-f6d138e3b171 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.039585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104a5e7-60d5-469f-98bd-8217ef4528da None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.068312 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.068936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.069213 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.069998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.073039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.073641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.073923 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.074194 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.074854 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:17.075155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.076591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.079571 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:17.084299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 73/149] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:17.084684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:17.086576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:17.087742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 77/150] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:17.091704 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.091815 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.166100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.189638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dd872-e03c-45c3-9b4f-0c54be7bfc11 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.189802 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da019227-03ed-4c30-92a7-ccffcbd6316e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.214883 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.223074 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.223715 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.223836 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.223994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.224264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] PATH_INFO: `/v3/users/eee6f083f9a24ff0adf9592a15c77cd4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.224455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.224736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.225012 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.226122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.227608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.229154 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 74/151] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:17.229905 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.241454 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] RBAC: Authorizing `identity:get_user(user_id=eee6f083f9a24ff0adf9592a15c77cd4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.243324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.244999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 78/152] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:17 2022] GET /identity/v3/users/eee6f083f9a24ff0adf9592a15c77cd4 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:17.245739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.329006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b858-fecb-4ebc-969b-d4a9b9b97c8a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.342154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60efbc0-11fd-437a-ae26-df27ea4a35ec None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.361385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.361984 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.362265 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.362535 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.363184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:17.363513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.364872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.372567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 75/153] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:17.373344 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.374324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.374926 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.375202 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.375504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.376139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.376721 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.377553 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.462392 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 79/154] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:17 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:17.463175 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.480478 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f795e-0150-437c-a6d2-7c448051a936 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.513648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.524393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.525310 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.525741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.526183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.527582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.529226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.531943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 76/155] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:17 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:17.538006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.586952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292273e7-4422-45c1-9a9c-e5138d01b081 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.618426 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.625449 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.626482 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.626765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.627373 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] PATH_INFO: `/v3/users/eee6f083f9a24ff0adf9592a15c77cd4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.628688 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] RBAC: Authorizing `identity:get_user(user_id=eee6f083f9a24ff0adf9592a15c77cd4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.630174 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.633134 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 80/156] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:17 2022] GET /identity/v3/users/eee6f083f9a24ff0adf9592a15c77cd4 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:17.637976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.650769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54f7ca-1f3a-407b-8394-6af8801cce29 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.692178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.692888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.695138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.695138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/eee6f083f9a24ff0adf9592a15c77cd4/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.695494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada30b34a9604dc8b6e53ee6f59b5305, user_id=eee6f083f9a24ff0adf9592a15c77cd4, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.696931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.756038 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91970c85-006d-41a5-ad70-45cd685365ad None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.765642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.771882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 77/157] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:17 2022] PUT /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/eee6f083f9a24ff0adf9592a15c77cd4/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:21:17.800226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c60089-d953-4ca2-95c3-e40b740ec68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.801182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c60089-d953-4ca2-95c3-e40b740ec68d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.801600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c60089-d953-4ca2-95c3-e40b740ec68d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.802018 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c60089-d953-4ca2-95c3-e40b740ec68d None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.803696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c60089-d953-4ca2-95c3-e40b740ec68d None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.805519 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c60089-d953-4ca2-95c3-e40b740ec68d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:17.807528 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 81/158] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:17 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:17.813721 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:17.936157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293c507-31c7-48ee-8582-99600f5a4878 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:17.969280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:17.974250 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fe1a2-5b20-4013-92d3-2c8d6c51fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:17.974916 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe1a2-5b20-4013-92d3-2c8d6c51fe22 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:17.975191 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe1a2-5b20-4013-92d3-2c8d6c51fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:17.976046 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fe1a2-5b20-4013-92d3-2c8d6c51fe22 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/eee6f083f9a24ff0adf9592a15c77cd4/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:17.977758 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fe1a2-5b20-4013-92d3-2c8d6c51fe22 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada30b34a9604dc8b6e53ee6f59b5305, user_id=eee6f083f9a24ff0adf9592a15c77cd4, role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:17.979186 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fe1a2-5b20-4013-92d3-2c8d6c51fe22 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:18.051632 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 78/159] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:17 2022] PUT /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/eee6f083f9a24ff0adf9592a15c77cd4/roles/517b2b1aaac34e92b002aec2385431d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:21:19.798246 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3085645-f52e-428f-9840-6b1cb00d8d3d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:19.798756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3085645-f52e-428f-9840-6b1cb00d8d3d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:19.798756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3085645-f52e-428f-9840-6b1cb00d8d3d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:19.800168 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 82/160] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:19.806062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032ce48-f986-422c-a3dc-fc5067e3442f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:19.806487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032ce48-f986-422c-a3dc-fc5067e3442f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:19.806698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032ce48-f986-422c-a3dc-fc5067e3442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:19.826016 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7032ce48-f986-422c-a3dc-fc5067e3442f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:19.827279 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7032ce48-f986-422c-a3dc-fc5067e3442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:19.900810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a065f0-94bd-4c52-9357-7b293d10bfe2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:19.901144 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a065f0-94bd-4c52-9357-7b293d10bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:19.901386 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a065f0-94bd-4c52-9357-7b293d10bfe2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:19.901765 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 79/161] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:19.902779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 83/162] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:19.908453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26b713-abd2-4f5d-a0ba-22e0c6565ac9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:19.908734 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26b713-abd2-4f5d-a0ba-22e0c6565ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:19.909005 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26b713-abd2-4f5d-a0ba-22e0c6565ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:19.927319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5d26b713-abd2-4f5d-a0ba-22e0c6565ac9 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:19.928496 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5d26b713-abd2-4f5d-a0ba-22e0c6565ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:19.978524 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a8a0-f6f2-4714-9a92-d974b0021ed1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:19.979167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a8a0-f6f2-4714-9a92-d974b0021ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:19.979706 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a8a0-f6f2-4714-9a92-d974b0021ed1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:19.981476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 84/163] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:19.986531 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c76a8-52a4-421e-8e84-56684ee02325 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:19.987037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c76a8-52a4-421e-8e84-56684ee02325 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:19.987551 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c76a8-52a4-421e-8e84-56684ee02325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:19.996294 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 80/164] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:20.008485 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ae9c76a8-52a4-421e-8e84-56684ee02325 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:20.009992 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ae9c76a8-52a4-421e-8e84-56684ee02325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:20.082581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 85/165] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:20.190064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84995570-e23f-446f-8afd-fd6a4a85d99c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.190425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84995570-e23f-446f-8afd-fd6a4a85d99c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.190616 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84995570-e23f-446f-8afd-fd6a4a85d99c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.192233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 81/166] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:20.197543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.198049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.198536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.205350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84995570-e23f-446f-8afd-fd6a4a85d99c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.218516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:20.220028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:20.274136 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:20.291910 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 86/167] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:20.300763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.329596 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84995570-e23f-446f-8afd-fd6a4a85d99c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.335623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:20.374131 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.374765 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.375044 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.375340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.385539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:20.390437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:20.391511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:20.392351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 82/168] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:20.396173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.405786 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74a1ca-55d6-4bc8-937e-3c18f024c6e0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.445340 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.446300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.446777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.447230 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.448610 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:20.450328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:20.452311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 87/169] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:20.457648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.509583 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f329b-879c-47f0-b762-ec05c09a97d6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.523561 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:20.545104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.545736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.546018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.546285 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.556863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:20.561929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:20.562806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 83/170] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:20.568397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.569297 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838597a-7deb-42aa-b0a6-5b3c27c20519 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.570374 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:20.602220 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.602829 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.603112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.603426 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.604092 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:20.604393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:20.605786 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:20.613675 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 88/171] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:20.618690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.668780 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7230-864b-416f-a881-51878fcc2734 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.701346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.701945 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.702223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.702494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.703134 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:20.704613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:20.715484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fa7a3-7951-4fe0-aed2-0c9063bd4929 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.748148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.748742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.749020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.749290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.749938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:20.750236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:20.751633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:20.759836 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 89/172] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:20.760602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.787213 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:20.799681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 84/173] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:20 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:20.802619 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.831835 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:20.869991 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63e2a9-6fe7-446a-b408-d7660e158389 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.910138 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.911148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.911503 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.911693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb951-525a-4c83-91b3-70dce19cd101 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:20.911794 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.924403 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:20.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:20.931287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:20.932131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 90/174] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:20.932853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:20.958334 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:20.958991 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:20.959273 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:20.959567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] PATH_INFO: `/v3/users/6f470d6a999e4a5b98f20f6ef815a1f0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:20.975342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] RBAC: Authorizing `identity:get_user(user_id=6f470d6a999e4a5b98f20f6ef815a1f0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:20.977936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:20.979340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 85/175] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/users/6f470d6a999e4a5b98f20f6ef815a1f0 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:20.980114 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.033221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.050042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e999f-c0b2-4746-bbb3-d67bdf3a53d9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.092261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.093188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.099216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.099216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.099216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] PATH_INFO: `/v3/users/eee6f083f9a24ff0adf9592a15c77cd4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.099216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] RBAC: Authorizing `identity:get_user(user_id=eee6f083f9a24ff0adf9592a15c77cd4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.099216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.099216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 91/176] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/users/eee6f083f9a24ff0adf9592a15c77cd4 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:21.099713 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.106364 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70194758-ca83-4af4-b567-6af4a6bf287e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.148654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.149341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.149764 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.150132 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.151004 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:21.151478 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.153039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.164511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 86/177] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:21.165494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.222484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420aa89-1fc1-4cff-8fd3-eef6b80dc899 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.270293 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.271479 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.271935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.273127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.275130 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.277413 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.279598 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 92/178] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:21.280246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc2d74-bf43-4724-8261-11a0e971e448 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.280716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.318829 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.322393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.323234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.323836 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.324312 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.349296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.403908 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ba5c-3f2a-47a4-a095-326acb174be6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.426454 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.427958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.453198 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.454392 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.455017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.455424 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] PATH_INFO: `/v3/users/eee6f083f9a24ff0adf9592a15c77cd4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.457348 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] RBAC: Authorizing `identity:get_user(user_id=eee6f083f9a24ff0adf9592a15c77cd4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.460356 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.462524 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 93/179] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/users/eee6f083f9a24ff0adf9592a15c77cd4 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:21.463284 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.543657 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 87/180] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:21.545113 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.571331 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.579987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7335120-a908-4bff-b05a-4ea970f0c365 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.616496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.617136 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.617248 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.617455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.617734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/6f470d6a999e4a5b98f20f6ef815a1f0/roles/d9150fbefb504f7d95662994e118df63` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.620068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada30b34a9604dc8b6e53ee6f59b5305, user_id=6f470d6a999e4a5b98f20f6ef815a1f0, role_id=d9150fbefb504f7d95662994e118df63)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.621625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.672115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3322b-584d-4026-ace8-cc98d4016a09 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.699048 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 94/181] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:21 2022] PUT /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/6f470d6a999e4a5b98f20f6ef815a1f0/roles/d9150fbefb504f7d95662994e118df63 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:21.700526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.720902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.722700 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.722700 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.725299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.861562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.863914 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.899053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3617f6-2a1e-4ea2-817c-679a055a5837 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:21.919330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.937336 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:21.937903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:21.938185 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:21.938458 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:21.939878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:21.941521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:21.944032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 95/182] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:21.949581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:21.972813 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:21.976202 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 88/183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:21.982006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:22.057212 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a61475-020a-4b2f-99b8-5453286dcefa None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:22.092215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:22.092972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:22.093353 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:22.093729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:22.094552 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305, user.id=eee6f083f9a24ff0adf9592a15c77cd4) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:22.094965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:22.096556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:22.103144 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 96/184] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/role_assignments?user.id=eee6f083f9a24ff0adf9592a15c77cd4&scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:22.114474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8583-b23a-4b51-8706-6c5d622b61ff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:22.159970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1b555-eb79-486a-a952-32a78631e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:22.161168 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b555-eb79-486a-a952-32a78631e243 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:22.161594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b555-eb79-486a-a952-32a78631e243 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:22.162011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b555-eb79-486a-a952-32a78631e243 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:22.164897 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1b555-eb79-486a-a952-32a78631e243 None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:22.166579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1b555-eb79-486a-a952-32a78631e243 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:22.168552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 89/185] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:21 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:22.174731 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:22.313964 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:22.343233 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51c718-0fb9-45ab-b988-43fc957fda46 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:22.391854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:22.392469 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:22.392749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:22.393018 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:22.393738 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] RBAC: Adding query filter params (role.id=517b2b1aaac34e92b002aec2385431d0, scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305, user.id=eee6f083f9a24ff0adf9592a15c77cd4) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:22.394040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:22.395507 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5771bef-6acc-4018-b1ba-2af16071bd91 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:22.407413 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 97/186] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:22 2022] GET /identity/v3/role_assignments?user.id=eee6f083f9a24ff0adf9592a15c77cd4&scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305&role.id=517b2b1aaac34e92b002aec2385431d0 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:23.230557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d92b22-5989-454a-9330-528fb2df54af None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.231079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d92b22-5989-454a-9330-528fb2df54af None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.231079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d92b22-5989-454a-9330-528fb2df54af None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.232480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 90/187] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:23.237568 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.238005 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.238147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.256696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:23.257866 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:23.295952 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad108-25fe-40fd-96d4-e17f68623e6d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.296276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad108-25fe-40fd-96d4-e17f68623e6d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.296506 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad108-25fe-40fd-96d4-e17f68623e6d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.297959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 91/188] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:21:23.302521 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.302819 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.303081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.323736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:23.324403 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 98/189] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:23.327628 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:23.336693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:23.392779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 92/190] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:23.590552 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:23.669931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:23.670144 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:23.692693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae56e6-494c-4879-a78e-84931d272be6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:23.730011 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:23.730791 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.731069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.731376 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.732757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:23.734277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:23.736169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 99/191] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:23.741185 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:23.778840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66cf95-1e16-4f99-a765-254c482cd861 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:23.813448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:23.814079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.814349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.814614 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.827230 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:23.832050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:23.832934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:23.833738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 93/192] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:23.838980 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:23.847053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db25e30-6471-4a98-85af-23cec4df19ab None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:23.848642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:23.882746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:23.883381 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.883933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.884577 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:23.886013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:23.938402 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca88af1-2448-44d1-8e50-1b1ea6ac81c5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:23.959587 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 100/193] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:23 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:21:23.973420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:23.974058 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:23.974342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:23.974621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:23.975938 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:23.976254 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:23.977724 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60698394-868c-4049-a495-3ce3a882cd6b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:23.987734 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 94/194] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:23.993974 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:23.996988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:24.109953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab341-00eb-465d-9d04-66cc257d74f5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:24.145636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:24.146273 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.146590 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.146816 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] PATH_INFO: `/v3/users/1000bd6927794234994d12f1a179d217` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.156931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb5ad0-04a1-4b41-9ff2-a1dfc51d53ea None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.157219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb5ad0-04a1-4b41-9ff2-a1dfc51d53ea None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.157502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb5ad0-04a1-4b41-9ff2-a1dfc51d53ea None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.158838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 95/195] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:24.162398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] RBAC: Authorizing `identity:get_user(user_id=1000bd6927794234994d12f1a179d217)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:24.164393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:24.165723 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.166009 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.166115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 101/196] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:23 2022] GET /identity/v3/users/1000bd6927794234994d12f1a179d217 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:24.166282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.173368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:24.186554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:24.187784 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:24.240577 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:24.265767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:24.273516 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 96/197] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:24.291277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298358de-0587-4ab6-b857-2151db5c14e3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:24.327154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:24.327823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.328106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.328377 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.329545 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:24.330994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:24.332942 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 102/198] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:24 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:24.339290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:24.449596 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25e2e0-923e-4b45-a6e0-5876ffa2a20a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:24.485901 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:24.486573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.486854 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.487129 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/1000bd6927794234994d12f1a179d217/roles/319c0aa665e04428bed4854c6664943a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.489094 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada30b34a9604dc8b6e53ee6f59b5305, user_id=1000bd6927794234994d12f1a179d217, role_id=319c0aa665e04428bed4854c6664943a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:24.490532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:24.559615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:24.565628 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 97/199] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:24 2022] PUT /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/users/1000bd6927794234994d12f1a179d217/roles/319c0aa665e04428bed4854c6664943a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:24.686854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0121c-3c3c-4057-9198-3f1e6c2f565b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:24.718104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:24.727820 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:24.728464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.728812 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.729091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.741148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:24.746013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:24.746904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:24.747752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 103/200] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:24.758317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:24.822535 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:24.865692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d589d-87e9-455c-b2b9-21989d1a6ca3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:24.901247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:24.901861 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:24.902143 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:24.902412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:24.903075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:24.903395 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:24.904753 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:24.914399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 98/201] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:24.920393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:25.028669 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835f5fd-af8e-4967-a890-9568d4793ce7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:25.064161 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:25.064786 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:25.065063 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:25.065332 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] PATH_INFO: `/v3/users/6f470d6a999e4a5b98f20f6ef815a1f0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:25.066441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] RBAC: Authorizing `identity:get_user(user_id=6f470d6a999e4a5b98f20f6ef815a1f0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:25.067927 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:25.069707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 104/202] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:24 2022] GET /identity/v3/users/6f470d6a999e4a5b98f20f6ef815a1f0 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:25.076541 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:25.183845 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628771-6a96-473d-9255-4edc6519beee None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:25.204157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:25.222940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:25.223590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:25.223869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:25.224141 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:25.294499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:25.295899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:25.396258 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 99/203] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:25.401692 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:25.461468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:25.507570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ada38-56c9-4d09-b764-5000692d473c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:25.541637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991fb25-b9d7-4179-96cb-c17a9532fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:25.542252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991fb25-b9d7-4179-96cb-c17a9532fe13 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:25.542531 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991fb25-b9d7-4179-96cb-c17a9532fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:25.542803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991fb25-b9d7-4179-96cb-c17a9532fe13 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:25.543892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991fb25-b9d7-4179-96cb-c17a9532fe13 None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:25.545436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991fb25-b9d7-4179-96cb-c17a9532fe13 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:25.547161 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 105/204] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:25 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:25.552875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:25.575531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:25.654668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1c4d6-83fa-4ef4-9c7a-847e3bbb491d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:25.688959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:25.689564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:25.689838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:25.690105 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:25.690816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] RBAC: Adding query filter params (role.id=d9150fbefb504f7d95662994e118df63, scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305, user.id=6f470d6a999e4a5b98f20f6ef815a1f0) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:25.691118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:25.692567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c87869-1fd8-4ebc-aa2c-0a6c4a7f824b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:25.698422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 100/205] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:25 2022] GET /identity/v3/role_assignments?user.id=6f470d6a999e4a5b98f20f6ef815a1f0&scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305&role.id=d9150fbefb504f7d95662994e118df63 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:26.289150 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd55d0-c833-4969-845a-c0dd434bc561 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.289479 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd55d0-c833-4969-845a-c0dd434bc561 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.289785 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd55d0-c833-4969-845a-c0dd434bc561 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.291266 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 106/206] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:26.296418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.296852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.297020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.314953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:26.316153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:26.366504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab294-48ce-4cb6-bf96-3d9bef29667a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.366781 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab294-48ce-4cb6-bf96-3d9bef29667a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.367056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab294-48ce-4cb6-bf96-3d9bef29667a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.368459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 107/207] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:26.376207 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.376495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.376770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.387100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 101/208] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:26.395270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:26.396502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:26.469204 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 108/209] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:26.653400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:26.669720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:26.737515 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:26.755534 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08aac-4395-4621-a507-2185c5496e3e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:26.800848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:26.809807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:26.810443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.810713 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.810978 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.821721 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:26.826443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:26.827358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:26.828350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 102/210] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:26.833229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:26.853135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e33ab-f296-492c-9e22-79879c39f348 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:26.888344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd58ce8-f1a3-4085-847d-60c8f91c955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:26.888977 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd58ce8-f1a3-4085-847d-60c8f91c955b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.889260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd58ce8-f1a3-4085-847d-60c8f91c955b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.889536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd58ce8-f1a3-4085-847d-60c8f91c955b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.890626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd58ce8-f1a3-4085-847d-60c8f91c955b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:26.892167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd58ce8-f1a3-4085-847d-60c8f91c955b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:26.893735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 109/211] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:26.896512 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63373bd9-56f8-487e-9185-024a0845d775 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.896840 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63373bd9-56f8-487e-9185-024a0845d775 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.897069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63373bd9-56f8-487e-9185-024a0845d775 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.898367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 110/212] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:26.899078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63373bd9-56f8-487e-9185-024a0845d775 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:26.912191 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:26.937646 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615eb01-faf8-44f9-bdd7-8a79a96b2214 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:26.970130 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:26.970733 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.971013 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.971286 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.971971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:26.972273 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:26.973651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2697a3-ce2d-446e-a803-4c9b41d8429b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:26.981982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 103/213] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:26.983106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:26.983426 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:26.983702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:26.999523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63373bd9-56f8-487e-9185-024a0845d775 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.000705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:27.002011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:27.022188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:27.033199 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:27.035620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:27.035620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:27.035620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:27.035620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:27.036627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:27.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 104/214] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:27.068598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:27.098014 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 111/215] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:26 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:21:27.138568 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:27.176798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6530703-f4f0-4fe1-acef-d4621a22fa07 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.213959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:27.214858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:27.215282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:27.215727 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] PATH_INFO: `/v3/users/04d7941ed41d4763b71bcda6af16612e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:27.230904 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] RBAC: Authorizing `identity:get_user(user_id=04d7941ed41d4763b71bcda6af16612e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:27.232744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:27.234591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 105/216] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:27 2022] GET /identity/v3/users/04d7941ed41d4763b71bcda6af16612e => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:27.240497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:27.336323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:27.345529 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef944394-3101-45d2-b2b3-0e99946b039c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.383617 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:27.384238 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:27.384519 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:27.384792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:27.385517 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] RBAC: Adding query filter params (role.id=d9150fbefb504f7d95662994e118df63, user.id=04d7941ed41d4763b71bcda6af16612e, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:27.385827 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:27.387274 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:27.393009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 112/217] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:21:27 2022] GET /identity/v3/role_assignments?user.id=04d7941ed41d4763b71bcda6af16612e&scope.system=all&role.id=d9150fbefb504f7d95662994e118df63 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:27.464620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1dff6-8c8c-44f4-b223-c7f9a11c21ba None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.492706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:27.505705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:27.506366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:27.506767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:27.507038 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:27.518919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:27.524455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:27.525401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:27.526205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 106/218] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:27.531940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:27.583735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:27.643864 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef2f8-db29-495b-a9fa-ff5f30732895 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.679627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:27.680234 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:27.680512 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:27.680778 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:27.681431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:27.681726 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:27.683104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:27.691667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 113/219] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:27.697923 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:27.805174 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b405e-9ce3-4328-b001-00ad03de27f2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.841816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:27.842792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:27.843272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:27.843792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] PATH_INFO: `/v3/users/1000bd6927794234994d12f1a179d217` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:27.845171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] RBAC: Authorizing `identity:get_user(user_id=1000bd6927794234994d12f1a179d217)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:27.846806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:27.848653 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 107/220] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:27 2022] GET /identity/v3/users/1000bd6927794234994d12f1a179d217 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:27.856021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:27.964042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f921a7-87a7-4cde-ab4a-8920daef104b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:27.966415 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:28.004923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:28.005780 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:28.006231 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:28.007202 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:28.080240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:28.081930 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:28.187951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 114/221] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:28.196475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:28.253741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:28.302361 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e56c3a-9bb4-4b96-b09c-1afcf8a04beb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:28.340000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3f9dc-d1d2-444c-9d47-cd7e079de1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:28.340696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3f9dc-d1d2-444c-9d47-cd7e079de1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:28.340906 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3f9dc-d1d2-444c-9d47-cd7e079de1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:28.341176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3f9dc-d1d2-444c-9d47-cd7e079de1f9 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:28.342290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3f9dc-d1d2-444c-9d47-cd7e079de1f9 None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:28.343821 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3f9dc-d1d2-444c-9d47-cd7e079de1f9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:28.345432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 108/222] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:28 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:28.351544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:28.354892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:28.460747 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4fb1b-2cf1-4bed-8fa7-843a12ef7da6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:28.495616 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:28.496221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:28.496507 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:28.496785 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:28.497509 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] RBAC: Adding query filter params (role.id=319c0aa665e04428bed4854c6664943a, scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305, user.id=1000bd6927794234994d12f1a179d217) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:28.497820 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:28.499274 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c29092-b2f4-4c8a-bc70-bdcd0c02dcb4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:28.505163 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 115/223] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:21:28 2022] GET /identity/v3/role_assignments?user.id=1000bd6927794234994d12f1a179d217&scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305&role.id=319c0aa665e04428bed4854c6664943a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:29.484395 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd83cf-9b69-4471-ab04-a7788682cb59 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.484900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd83cf-9b69-4471-ab04-a7788682cb59 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.484973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd83cf-9b69-4471-ab04-a7788682cb59 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:29.486264 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 109/224] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:29.491728 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7b74-60c9-400e-a05f-45d14aa1a778 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.492163 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7b74-60c9-400e-a05f-45d14aa1a778 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.492271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f7b74-60c9-400e-a05f-45d14aa1a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:29.501116 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5a903-ce78-4177-b22c-88db01c72e88 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.501401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5a903-ce78-4177-b22c-88db01c72e88 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.501678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5a903-ce78-4177-b22c-88db01c72e88 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:29.502977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 110/225] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:21:29.508272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba168c-d704-4e84-a972-19143bcf3f7e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.508560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba168c-d704-4e84-a972-19143bcf3f7e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.508835 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba168c-d704-4e84-a972-19143bcf3f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:29.510600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-af7f7b74-60c9-400e-a05f-45d14aa1a778 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:29.511806 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-af7f7b74-60c9-400e-a05f-45d14aa1a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:29.527030 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f8ba168c-d704-4e84-a972-19143bcf3f7e None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:29.528272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f8ba168c-d704-4e84-a972-19143bcf3f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:29.582338 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:29.589260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 116/226] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:29.605016 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 111/227] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:29.653143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c881d1-6fa5-4a99-bf4e-99c40253dd15 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.653451 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c881d1-6fa5-4a99-bf4e-99c40253dd15 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.653690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c881d1-6fa5-4a99-bf4e-99c40253dd15 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:29.655147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 117/228] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:29.660044 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.660277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.660801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:29.679093 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:29.680285 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:29.725696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:29.745505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 112/229] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:29.856259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c881d1-6fa5-4a99-bf4e-99c40253dd15 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:29.872874 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:29.958614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c881d1-6fa5-4a99-bf4e-99c40253dd15 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:29.979259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a190373-9204-420e-b459-e9099d38e8c0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:29.992944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:29.993720 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:29.994357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:29.994633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.004447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:30.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:30.010611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:30.011488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 118/230] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:30.012137 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.017818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.018452 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.018733 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.019007 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.020203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.021627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.027532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 113/231] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:30.027532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.070363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:30.071606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:30.110396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465df4c2-9e6d-483d-9606-4701d60582bb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.119633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696570a-3a34-42d6-81c1-09375ad2188a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.143024 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.143662 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.143940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.144211 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.152655 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.153258 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.153538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.153807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.153900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:30.154459 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:30.154758 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.156146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:30.159025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:30.159944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:30.161016 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 119/232] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:30.163288 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.171469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 114/233] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:30.171469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.258380 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c378b-da37-478a-9486-35ac62500d95 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.265156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5271a-1565-41d8-951e-1da68fd7783a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.274804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:30.275719 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:30.291018 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.291654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.291937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.292212 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] PATH_INFO: `/v3/groups` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.292865 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.294265 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.297457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.298049 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.298326 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.298598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.299242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:30.299568 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.300913 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.310140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 115/234] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:30.310911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.347499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 120/235] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Oct 17 05:21:30 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:30.348308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.431876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f92ae-d387-43f8-8648-d7129dd87fae None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.467255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6baebb-fe2c-4a30-b3f6-588b38162ba9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.468790 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.469510 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.469858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.470234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] PATH_INFO: `/v3/users/77e0d22633734a13881086accc1f8844` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.485439 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] RBAC: Authorizing `identity:get_user(user_id=77e0d22633734a13881086accc1f8844)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.486955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.488742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 116/236] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:30 2022] GET /identity/v3/users/77e0d22633734a13881086accc1f8844 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:30.495412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.510306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.510956 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.511138 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.511445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] PATH_INFO: `/v3/users/04d7941ed41d4763b71bcda6af16612e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.512588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] RBAC: Authorizing `identity:get_user(user_id=04d7941ed41d4763b71bcda6af16612e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.514071 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.515719 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 121/237] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:30 2022] GET /identity/v3/users/04d7941ed41d4763b71bcda6af16612e => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:30.522518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:30.529480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:30.575438 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:30.609326 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b6617-2c86-4ea9-92e6-9f87348aeb55 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.633532 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2f16d-d898-42aa-950a-2f388a725cfd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:30.644549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.645478 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.645925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.646203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.646933 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] RBAC: Adding query filter params (role.id=319c0aa665e04428bed4854c6664943a, user.id=77e0d22633734a13881086accc1f8844, scope.system=all) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:30.647477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.649290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984617d-0f62-4172-bcd8-b33d24766947 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.654294 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 117/238] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:21:30 2022] GET /identity/v3/role_assignments?user.id=77e0d22633734a13881086accc1f8844&scope.system=all&role.id=319c0aa665e04428bed4854c6664943a => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:30.669516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617f358-e670-480a-82d8-9819d0c8b857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:30.670191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f358-e670-480a-82d8-9819d0c8b857 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:30.670468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f358-e670-480a-82d8-9819d0c8b857 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:30.670734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f358-e670-480a-82d8-9819d0c8b857 None admin] PATH_INFO: `/v3/system/users/04d7941ed41d4763b71bcda6af16612e/roles/d9150fbefb504f7d95662994e118df63` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:30.672249 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f358-e670-480a-82d8-9819d0c8b857 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=04d7941ed41d4763b71bcda6af16612e, role_id=d9150fbefb504f7d95662994e118df63)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:30.673692 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f358-e670-480a-82d8-9819d0c8b857 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:30.800485 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 122/239] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Oct 17 05:21:30 2022] PUT /identity/v3/system/users/04d7941ed41d4763b71bcda6af16612e/roles/d9150fbefb504f7d95662994e118df63 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:21:32.538937 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e03be-e11c-486a-a40c-9467292779d0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:32.539469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e03be-e11c-486a-a40c-9467292779d0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:32.539543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e03be-e11c-486a-a40c-9467292779d0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:32.540970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 118/240] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:32.546069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226557-d786-4c5c-ae48-517c6c4f7bad None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:32.546474 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226557-d786-4c5c-ae48-517c6c4f7bad None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:32.546634 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226557-d786-4c5c-ae48-517c6c4f7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:32.565756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5a226557-d786-4c5c-ae48-517c6c4f7bad None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:32.566919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5a226557-d786-4c5c-ae48-517c6c4f7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:32.634466 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 123/241] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:32.800634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983350e3-b836-4981-b353-940459ef7f58 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:32.800811 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983350e3-b836-4981-b353-940459ef7f58 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:32.801090 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983350e3-b836-4981-b353-940459ef7f58 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:32.802456 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 119/242] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:32.808883 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a745-1f12-4e70-9f5c-ed8d65eb0e4a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:32.809168 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a745-1f12-4e70-9f5c-ed8d65eb0e4a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:32.809445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a745-1f12-4e70-9f5c-ed8d65eb0e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:32.829494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0484a745-1f12-4e70-9f5c-ed8d65eb0e4a None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:32.830716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0484a745-1f12-4e70-9f5c-ed8d65eb0e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:32.865909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:32.898927 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 124/243] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:32.904128 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983350e3-b836-4981-b353-940459ef7f58 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:32.922655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3004124-90ff-4d9c-b3fe-a6dd6219b8d7 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:32.922943 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3004124-90ff-4d9c-b3fe-a6dd6219b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:32.923216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3004124-90ff-4d9c-b3fe-a6dd6219b8d7 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:32.924756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 125/244] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:21:32.930065 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:32.930825 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:32.931392 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:32.949073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:32.950236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:33.003524 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:33.012258 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983350e3-b836-4981-b353-940459ef7f58 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.021096 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 126/245] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:33.046241 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.046860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.047135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.047435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.048495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.049912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.051685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 120/246] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:33.056881 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.151792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:33.155210 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bb7a-7c28-4f77-9d85-3a2348014a7f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.167857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.191665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.192281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.192561 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.192846 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] PATH_INFO: `/v3/groups` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.193490 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.195034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.267778 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b01105-de92-48d4-b907-013eb250e7b9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.300576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.301255 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.301531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.301852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.319399 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 127/247] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Oct 17 05:21:33 2022] POST /identity/v3/groups => generated 226 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:33.319399 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:33.319682 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:33.320588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:33.321622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 121/248] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:33.325153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.353545 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:33.429053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce90526-e92e-45ba-ae51-2719ce0dce6f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.442784 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37467564-d5a4-43bc-ab8e-0360472421e6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.468212 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.468857 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.469140 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.469416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.480366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:33.485671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:33.486534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:33.487354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:33.487964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.487964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.488112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 128/249] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:33.488179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.488179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.488179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:33.488179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.489502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.493845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.499542 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 122/250] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:33.504633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.518658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:33.608699 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:33.613382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da43fc9-78ed-4970-8255-c9ff714749ca None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.631696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18435d01-0b59-4c8d-b1bb-be5118cac499 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.655861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.657032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.657311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.657579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.658521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:33.658820 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.661432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.672962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 129/251] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:33.677722 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.678401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.678678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.678950 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] PATH_INFO: `/v3/users/77e0d22633734a13881086accc1f8844` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.680285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.681362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] RBAC: Authorizing `identity:get_user(user_id=77e0d22633734a13881086accc1f8844)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.682753 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.684909 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 123/252] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/users/77e0d22633734a13881086accc1f8844 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:33.691523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.785763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:33.808785 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848993f5-1b3c-4e85-b945-23cd5ee5de7d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.821554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce52d05-0632-4d00-8789-bf0627124f89 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:33.867065 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.867824 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.868107 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.868388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] PATH_INFO: `/v3/users/04d7941ed41d4763b71bcda6af16612e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.870650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] RBAC: Authorizing `identity:get_user(user_id=04d7941ed41d4763b71bcda6af16612e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.872065 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.872298 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648782d-3ca5-4601-b765-ccfa55fd81e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:33.873156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648782d-3ca5-4601-b765-ccfa55fd81e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:33.873640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648782d-3ca5-4601-b765-ccfa55fd81e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:33.873755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 130/253] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/users/04d7941ed41d4763b71bcda6af16612e => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:33.874208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648782d-3ca5-4601-b765-ccfa55fd81e3 None admin] PATH_INFO: `/v3/system/users/77e0d22633734a13881086accc1f8844/roles/319c0aa665e04428bed4854c6664943a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:33.875849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648782d-3ca5-4601-b765-ccfa55fd81e3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=77e0d22633734a13881086accc1f8844, role_id=319c0aa665e04428bed4854c6664943a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:33.877422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648782d-3ca5-4601-b765-ccfa55fd81e3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:33.880688 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:33.948157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 124/254] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Oct 17 05:21:33 2022] PUT /identity/v3/system/users/77e0d22633734a13881086accc1f8844/roles/319c0aa665e04428bed4854c6664943a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:34.005757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcc656-88f7-41b2-a8b6-f575680519de None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:34.046993 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:34.047651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:34.047931 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:34.048203 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:34.048936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] RBAC: Adding query filter params (role.id=d9150fbefb504f7d95662994e118df63, user.id=04d7941ed41d4763b71bcda6af16612e, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:34.049247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:34.050713 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337e3f-eaf2-4aaf-9e3c-832fad2e2fce None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:34.064954 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 131/255] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:21:33 2022] GET /identity/v3/role_assignments?user.id=04d7941ed41d4763b71bcda6af16612e&scope.system=all&role.id=d9150fbefb504f7d95662994e118df63 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:35.805085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630c83-04c3-45bb-931f-5eeca668111f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:35.806030 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630c83-04c3-45bb-931f-5eeca668111f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:35.806530 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630c83-04c3-45bb-931f-5eeca668111f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:35.808111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 125/256] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:35.813770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222332d1-030c-4648-a3ed-83bf56d329d0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:35.814240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222332d1-030c-4648-a3ed-83bf56d329d0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:35.814312 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222332d1-030c-4648-a3ed-83bf56d329d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:35.848285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-222332d1-030c-4648-a3ed-83bf56d329d0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:35.849502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-222332d1-030c-4648-a3ed-83bf56d329d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:35.866374 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:35.970333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 132/257] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:21:36.111975 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d3f41-6895-4b58-a7e3-592292363b40 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.112330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d3f41-6895-4b58-a7e3-592292363b40 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.112527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d3f41-6895-4b58-a7e3-592292363b40 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.114011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 126/258] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:36.131724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.131724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.131724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.160262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:36.162098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:36.173600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 133/259] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:21:36.570354 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d3f41-6895-4b58-a7e3-592292363b40 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.589424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d3f41-6895-4b58-a7e3-592292363b40 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.596776 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.597656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.598126 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.598585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.612456 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:36.617730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:36.618577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:36.619396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:36.621913 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 127/260] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:36.627446 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.641936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0608d6d-7b29-469b-a8e9-e28001c7df94 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.652354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.652977 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.653257 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.653532 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.654208 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:36.654511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.656339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:21:36.656339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 05:21:36.656339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: Oct 17 05:21:36.657249 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.672758 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 134/261] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:36.674436 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.677579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.684652 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:36.687872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42910a-868e-414e-96ab-b2f0e7224ec1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.689777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef9f16-393d-418d-9c1a-673a97b4d93f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.695081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.696066 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.696501 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.696933 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.699260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.699722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.700003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.700274 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.714339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.715088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:36.718480 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:21:36.718480 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 05:21:36.718480 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: Oct 17 05:21:36.720049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.725561 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 135/262] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:36.726899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:36.727981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:36.728925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:36.731138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 128/263] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:36.732970 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.739062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.751012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86f99c-3580-4997-9cb1-2136898b2cbd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.755949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4222d-0833-45bb-b484-31327163c2cc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.762399 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.763778 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.764515 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.765875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.766344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.767740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.768539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.769000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.769411 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.770694 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:36.772914 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.773472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.775492 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:21:36.775492 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 05:21:36.775492 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: Oct 17 05:21:36.777679 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 136/264] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:36.778801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.783656 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.798784 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 129/265] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:36.801307 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2162e-37c4-40d5-82f2-ed7489e89e15 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.804569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.811973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.813686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.814369 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.816300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de/groups/c0199018169a443f855bb26784cc7eeb/roles/d9150fbefb504f7d95662994e118df63` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.821085 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17423390f676473593042b205bc4d4de, group_id=c0199018169a443f855bb26784cc7eeb, role_id=d9150fbefb504f7d95662994e118df63)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.822259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:21:36.822259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 05:21:36.822259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: Oct 17 05:21:36.824542 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6253175-0019-4a21-821f-e3b7d70da6b2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.824685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee87c6-46c7-47c9-8167-89ba53b6cef5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.833133 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.833946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.834428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.834863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.836125 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.837275 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:21:36.837275 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 05:21:36.837275 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: Oct 17 05:21:36.838786 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.841176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 130/266] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:36.846569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.857749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0decc-6837-4eb7-9107-43cee20ded2f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.862787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.863700 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.864199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.864606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.866022 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.868443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.870489 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 131/267] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:21:36 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:36.876293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:36.888104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfeeb64-305c-4af8-872c-ca6759a96478 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:36.894227 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:36.895213 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.895743 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.896212 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de/groups/c0199018169a443f855bb26784cc7eeb/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.898875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17423390f676473593042b205bc4d4de, group_id=c0199018169a443f855bb26784cc7eeb, role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:36.900263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:21:36.900263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 05:21:36.900263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: Oct 17 05:21:36.903820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239ff0d-5b7d-46ea-bb41-3c8b6ae93699 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:36.930989 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 137/268] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:21:36 2022] PUT /identity/v3/projects/17423390f676473593042b205bc4d4de/groups/c0199018169a443f855bb26784cc7eeb/roles/d9150fbefb504f7d95662994e118df63 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:36.932228 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190097d-106f-4e0e-b991-ddad8c899842 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.932516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190097d-106f-4e0e-b991-ddad8c899842 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.932809 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190097d-106f-4e0e-b991-ddad8c899842 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.934283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 138/269] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:36.940263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44913bec-5c01-4a09-a91b-37e1c2537363 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:36.940566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44913bec-5c01-4a09-a91b-37e1c2537363 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:36.940838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44913bec-5c01-4a09-a91b-37e1c2537363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:36.964440 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-44913bec-5c01-4a09-a91b-37e1c2537363 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:36.965665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-44913bec-5c01-4a09-a91b-37e1c2537363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:36.987710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 132/270] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:21:36 2022] PUT /identity/v3/projects/17423390f676473593042b205bc4d4de/groups/c0199018169a443f855bb26784cc7eeb/roles/517b2b1aaac34e92b002aec2385431d0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:21:37.054278 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 139/271] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 05:21:37.293171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fea8-a1c4-4522-99e4-aa2cf2e4b780 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.293373 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fea8-a1c4-4522-99e4-aa2cf2e4b780 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.293648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fea8-a1c4-4522-99e4-aa2cf2e4b780 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.295332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 133/272] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:37.300727 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.301013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.301285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.323776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:37.325029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:37.386675 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:37.398021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 140/273] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:37.482372 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fea8-a1c4-4522-99e4-aa2cf2e4b780 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:37.572986 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fea8-a1c4-4522-99e4-aa2cf2e4b780 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:37.580839 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:37.581481 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.581758 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.582030 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.598237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:37.607900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:37.608903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:37.609714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:37.610331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 134/274] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:21:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 128 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:37.616449 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:37.632482 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756edaf8-9ee9-4d85-ba31-b396fe6e98b6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:37.639463 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:37.640125 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.640403 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.640705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.641377 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:37.641716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:37.643395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:37.653166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 141/275] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:21:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:37.658822 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:37.674149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc437a-8a7a-4a50-8030-424e363315f1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:37.681280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:37.681916 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.682193 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.682455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.683617 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:37.685659 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:37.687272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 135/276] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:21:37 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:37.693181 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:37.705706 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943052f-6600-41dc-bf5b-9c8cb4c73f86 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:37.711632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ffb97-48c0-4b11-82f4-75d5c307eaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:37.712350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ffb97-48c0-4b11-82f4-75d5c307eaab admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.712673 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ffb97-48c0-4b11-82f4-75d5c307eaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.712940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ffb97-48c0-4b11-82f4-75d5c307eaab admin admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.714109 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ffb97-48c0-4b11-82f4-75d5c307eaab admin admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:37.716388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ffb97-48c0-4b11-82f4-75d5c307eaab admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:37.718188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 142/277] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:21:37 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:37.724145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:37.737162 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a33ba3-1e78-480e-a2ed-388554010bc2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:37.742589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:37.743270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.743649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.743922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/groups/c0199018169a443f855bb26784cc7eeb/roles/d9150fbefb504f7d95662994e118df63` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.745960 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada30b34a9604dc8b6e53ee6f59b5305, group_id=c0199018169a443f855bb26784cc7eeb, role_id=d9150fbefb504f7d95662994e118df63)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:37.749033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:37.877445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b441-0f4b-4027-97e7-21f4bdb3e869 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.877951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b441-0f4b-4027-97e7-21f4bdb3e869 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.877951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b441-0f4b-4027-97e7-21f4bdb3e869 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.879345 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 143/278] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:37.886977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 136/279] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:21:37 2022] PUT /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/groups/c0199018169a443f855bb26784cc7eeb/roles/d9150fbefb504f7d95662994e118df63 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:37.888645 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:37.888983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:37.889256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:37.901743 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:37.928644 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:37.930118 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:38.052485 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:38.069334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 144/280] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:21:38.074461 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e8367-7f74-4682-a14d-faae84e9910d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.121172 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.122133 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.122554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.122961 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.135186 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:38.144739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:38.145863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:21:38.146891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:38.148866 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 137/281] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:38.160760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.305534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc81c2-cb48-45a5-a536-518e909de806 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.330307 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:38.347638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.348259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.348548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.348825 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.349497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:38.349802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.351736 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.363826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 145/282] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:38.370014 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.544490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825e88-0758-4260-bd3f-ca0d82092f54 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.563331 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.588281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239992c6-883d-45ad-891a-386db02bbf79 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.596442 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.597090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.597489 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.597875 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.598834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.599506 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.599820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.600748 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] PATH_INFO: `/v3/users/77e0d22633734a13881086accc1f8844` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.604696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] RBAC: Authorizing `identity:get_user(user_id=77e0d22633734a13881086accc1f8844)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.606454 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.608107 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 138/283] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/users/77e0d22633734a13881086accc1f8844 => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:38.614589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.632037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:38.637425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:38.638311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:38.639100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:38.640223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 146/284] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 76 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:38.644600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.652639 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:38.658782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8540e-2b92-4d78-aadf-8ed130a01aae admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.670877 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.671240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.671537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.671810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.672545 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:38.672849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.674465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 147/285] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:38.699847 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.718025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5da4fc-9b83-4737-94ef-3246c7d685c2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.732973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.733626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.733905 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.734178 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.735283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.737225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.738790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 148/286] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:38.744344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.761702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ba9dd-95ec-477a-b1ef-b39b6d9b00b5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.770673 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.771340 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.771633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.772425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.773617 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.775708 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.782023 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 149/287] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:38.783199 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:38.801690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09265bf-c0d7-4202-8b78-26aca26a72d4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.812856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0aa40-22de-483d-b898-b9facf406bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.813527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0aa40-22de-483d-b898-b9facf406bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.813856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0aa40-22de-483d-b898-b9facf406bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.814142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0aa40-22de-483d-b898-b9facf406bf5 admin admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/groups/c0199018169a443f855bb26784cc7eeb/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.816310 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0aa40-22de-483d-b898-b9facf406bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada30b34a9604dc8b6e53ee6f59b5305, group_id=c0199018169a443f855bb26784cc7eeb, role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.819563 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0aa40-22de-483d-b898-b9facf406bf5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.824794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b09409-5e0a-4569-af91-eb365c75c7ac None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:38.881992 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:38.883075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.883636 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.884105 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.885077 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] RBAC: Adding query filter params (role.id=319c0aa665e04428bed4854c6664943a, user.id=77e0d22633734a13881086accc1f8844, scope.system=all) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:38.885656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:38.890458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a71342-d0ca-44d7-addc-299c08753c4e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:38.907788 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 139/288] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:21:38 2022] GET /identity/v3/role_assignments?user.id=77e0d22633734a13881086accc1f8844&scope.system=all&role.id=319c0aa665e04428bed4854c6664943a => generated 510 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:38.910282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 150/289] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:21:38 2022] PUT /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305/groups/c0199018169a443f855bb26784cc7eeb/roles/517b2b1aaac34e92b002aec2385431d0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:21:38.995124 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8bb06-dad1-467d-a380-59db7cd5349a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:38.995314 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8bb06-dad1-467d-a380-59db7cd5349a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:38.995603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8bb06-dad1-467d-a380-59db7cd5349a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:38.997343 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 140/290] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:39.004938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:39.005250 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:39.005501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:39.031159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:39.032420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:39.108892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 151/291] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:21:39.431081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8bb06-dad1-467d-a380-59db7cd5349a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:39.445247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8bb06-dad1-467d-a380-59db7cd5349a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:39.449575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:39.450203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:39.450479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:39.450749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:39.462820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:39.473063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:39.474032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:39.474924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 141/292] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:39.480547 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:39.494925 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa408b7c-d36b-449c-9a35-1269617c6528 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:39.503413 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:39.504033 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:39.504313 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:39.504584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:39.505256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:39.505556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:39.507358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:39.522922 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 152/293] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:39.528681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:39.542656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110492d-2b32-4dbc-845a-f12e4f9f6d63 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:39.551221 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:39.551906 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:39.552190 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:39.552454 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] PATH_INFO: `/v3/groups/140f74f8a4834b04afc48782cc13ad46` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:39.558820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] RBAC: Authorizing `identity:get_group(group_id=140f74f8a4834b04afc48782cc13ad46)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:39.560732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:39.562277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 142/294] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:21:39 2022] GET /identity/v3/groups/140f74f8a4834b04afc48782cc13ad46 => generated 229 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:39.567947 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:39.583812 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9553c-c7be-4cd4-81e0-ba18ccde7024 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:39.591540 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84447e8-786c-450c-ab46-473fad17993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:39.592180 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84447e8-786c-450c-ab46-473fad17993a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:39.592517 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84447e8-786c-450c-ab46-473fad17993a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:39.592792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84447e8-786c-450c-ab46-473fad17993a admin admin] PATH_INFO: `/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:39.593968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84447e8-786c-450c-ab46-473fad17993a admin admin] RBAC: Authorizing `identity:get_project(project_id=29daf56ceab84ba1bd5c5b6f664b751b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:39.595908 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84447e8-786c-450c-ab46-473fad17993a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:39.597696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 153/295] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:21:39 2022] GET /identity/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b => generated 344 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:39.604953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:39.614887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c594f-260a-4633-a2c9-6c40d021bec7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:39.621395 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d35d60-6bb3-4e11-8c55-4b694bbf42f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29daf56ceab84ba1bd5c5b6f664b751b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:39.622058 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35d60-6bb3-4e11-8c55-4b694bbf42f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:39.622336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35d60-6bb3-4e11-8c55-4b694bbf42f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:39.622608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35d60-6bb3-4e11-8c55-4b694bbf42f9 admin admin] PATH_INFO: `/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b/groups/140f74f8a4834b04afc48782cc13ad46/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:39.624486 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d35d60-6bb3-4e11-8c55-4b694bbf42f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29daf56ceab84ba1bd5c5b6f664b751b, group_id=140f74f8a4834b04afc48782cc13ad46, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:39.627098 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d35d60-6bb3-4e11-8c55-4b694bbf42f9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:39.720399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 143/296] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:21:39 2022] PUT /identity/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b/groups/140f74f8a4834b04afc48782cc13ad46/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:40.278929 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674044a-901c-4f03-8395-56459e6682da None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:40.279512 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674044a-901c-4f03-8395-56459e6682da None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:40.279512 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674044a-901c-4f03-8395-56459e6682da None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:40.281231 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 154/297] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:40.287807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4e35-868e-4d0b-a302-53393c0031b4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:40.288442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4e35-868e-4d0b-a302-53393c0031b4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:40.288442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4e35-868e-4d0b-a302-53393c0031b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:40.311412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-926b4e35-868e-4d0b-a302-53393c0031b4 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:40.312639 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-926b4e35-868e-4d0b-a302-53393c0031b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:40.353668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:40.391403 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 144/298] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:40.422875 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d63ca-69a7-47aa-aa4e-2ea34bcdd8f9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:40.423208 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d63ca-69a7-47aa-aa4e-2ea34bcdd8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:40.423438 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d63ca-69a7-47aa-aa4e-2ea34bcdd8f9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:40.424782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 155/299] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:40.432733 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced826bb-cb15-4096-a95a-1bf55135a8af None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:40.433017 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced826bb-cb15-4096-a95a-1bf55135a8af None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:40.433293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced826bb-cb15-4096-a95a-1bf55135a8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:40.455792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ced826bb-cb15-4096-a95a-1bf55135a8af None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:40.456954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ced826bb-cb15-4096-a95a-1bf55135a8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:40.532413 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 145/300] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:40.990313 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ff470-f0b3-4aa7-8bbd-a201f0aedfe5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:40.991338 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ff470-f0b3-4aa7-8bbd-a201f0aedfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:40.991831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ff470-f0b3-4aa7-8bbd-a201f0aedfe5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:40.993631 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 156/301] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:40.999216 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:40.999760 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.000239 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.006412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ff470-f0b3-4aa7-8bbd-a201f0aedfe5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.039138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:41.040389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:41.119092 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:41.137109 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 146/302] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:41.138244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.175506 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ff470-f0b3-4aa7-8bbd-a201f0aedfe5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.200279 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:41.247684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.248485 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.248768 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.249041 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.276656 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:41.283777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:41.285075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:41.286133 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 157/303] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:41.291926 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.329889 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54324898-e1fd-469d-9d77-dd26242e8595 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.405827 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.406485 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.406769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.407042 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.440338 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:41.449951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:41.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:41.452069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:41.457156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 147/304] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:41.469595 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.510037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4b0e-facf-46dc-928b-c28d843ab7af None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.550508 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.551677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.553513 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.553803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.554476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:41.554784 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:41.556225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:41.557622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:41.564992 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 158/305] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:41.565822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.603268 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:41.617040 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda6e0-2360-496f-b8f5-0a12ba637677 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.659009 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.659562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.660087 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.660447 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.661755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:41.661755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:41.663126 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:41.675328 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 148/306] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:41.676219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.702584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9730e7-40be-4892-b06d-373da70ddd83 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.757350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.757996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.758328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.758608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.774798 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:41.781527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:41.782459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:41.783754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 159/307] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:41.784492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:41.814022 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71806bfd-84ec-4e19-9dc3-4e6c844face9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.848911 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:41.864350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.865890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.866179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.866534 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.891155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:41.905682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119081-21ae-4e21-be0b-f559b2b74572 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:41.952451 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899c775-724b-400f-a06b-89b8319e0e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:41.952843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:41.953066 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899c775-724b-400f-a06b-89b8319e0e6f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:41.953351 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899c775-724b-400f-a06b-89b8319e0e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:41.953625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899c775-724b-400f-a06b-89b8319e0e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:41.954323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.046481 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899c775-724b-400f-a06b-89b8319e0e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.048040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899c775-724b-400f-a06b-89b8319e0e6f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.085894 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 149/308] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:42.087161 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.115562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.149398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.165148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 160/309] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:42.166609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d7e36-eae4-461e-b232-4da1ba56be77 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.166959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d7e36-eae4-461e-b232-4da1ba56be77 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.167306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d7e36-eae4-461e-b232-4da1ba56be77 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.168633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 161/310] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:42.169457 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d7e36-eae4-461e-b232-4da1ba56be77 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.206984 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59081b-9181-4b86-ad8f-5ce89ee28000 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.252477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.253101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.253384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.253830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.254517 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:42.255983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.257011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.269748 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 150/311] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:42.270381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.299730 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d7e36-eae4-461e-b232-4da1ba56be77 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.347400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a9202-eb62-4513-98e9-8d90c2cee70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.348347 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a9202-eb62-4513-98e9-8d90c2cee70f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.348812 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a9202-eb62-4513-98e9-8d90c2cee70f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.349252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a9202-eb62-4513-98e9-8d90c2cee70f None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.350596 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a9202-eb62-4513-98e9-8d90c2cee70f None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.352319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a9202-eb62-4513-98e9-8d90c2cee70f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.354137 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 162/312] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:42.355459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.355914 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.356359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.375586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:42.378300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:42.396817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.402689 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38ecfd-59c1-42bf-9904-e613e8b8501a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.422657 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.448388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.449024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.450207 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.450487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] PATH_INFO: `/v3/projects/17423390f676473593042b205bc4d4de` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.451928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] RBAC: Authorizing `identity:get_project(project_id=17423390f676473593042b205bc4d4de)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.453368 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.455510 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 151/313] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/projects/17423390f676473593042b205bc4d4de => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:42.456354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 163/314] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:42.456763 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.456958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.578049 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58727e7a-a102-430c-8417-8649a345c326 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.581011 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95f24b-7fec-4515-b585-13ee874e1121 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.626074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81904e2-5fc6-4847-9c24-61ccea625fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.627878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81904e2-5fc6-4847-9c24-61ccea625fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.627878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81904e2-5fc6-4847-9c24-61ccea625fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.627878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81904e2-5fc6-4847-9c24-61ccea625fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.628126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65da5e-fba6-4f09-a567-f55b2003fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.628417 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65da5e-fba6-4f09-a567-f55b2003fbea None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.628696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65da5e-fba6-4f09-a567-f55b2003fbea None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.628968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65da5e-fba6-4f09-a567-f55b2003fbea None admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.630056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65da5e-fba6-4f09-a567-f55b2003fbea None admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.633191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65da5e-fba6-4f09-a567-f55b2003fbea None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.634901 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 164/315] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:42.636093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7ff76-8b62-4292-aa48-dd3505b94b28 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.636372 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7ff76-8b62-4292-aa48-dd3505b94b28 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.636642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7ff76-8b62-4292-aa48-dd3505b94b28 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.637987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 165/316] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:42.638522 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7ff76-8b62-4292-aa48-dd3505b94b28 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.673879 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.696953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.720867 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81904e2-5fc6-4847-9c24-61ccea625fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.722307 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81904e2-5fc6-4847-9c24-61ccea625fd9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.770892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7ff76-8b62-4292-aa48-dd3505b94b28 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.816124 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.816755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.817067 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.817341 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.818970 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.820434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.822415 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 166/317] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:42.823040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.844337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 152/318] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:42.846182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.846182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.846182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.865152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:42.866904 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:42.925875 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.938984 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbe281-5d63-4628-a1d0-0e634240aa64 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:42.958065 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 153/319] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:42.959763 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:42.974670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:42.986089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:42.987055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:42.987527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:42.987959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:42.988837 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] RBAC: Adding query filter params (group.id=c0199018169a443f855bb26784cc7eeb, role.id=517b2b1aaac34e92b002aec2385431d0, scope.project.id=17423390f676473593042b205bc4d4de) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:42.989355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:42.991588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:42.997821 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 167/320] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/role_assignments?group.id=c0199018169a443f855bb26784cc7eeb&scope.project.id=17423390f676473593042b205bc4d4de&role.id=517b2b1aaac34e92b002aec2385431d0 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:42.998848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.073355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c4f32-ec64-4872-8c1d-c4f07fa2309c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.116532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.117434 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.117858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.118272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.130286 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:43.136299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:43.137458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:43.138453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:43.140500 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 154/321] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:43.141542 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.165335 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148e508-3de4-462b-b272-af39e924f781 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.216883 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.217502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.217788 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.218053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.218780 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] RBAC: Adding query filter params (group.id=c0199018169a443f855bb26784cc7eeb, role.id=d9150fbefb504f7d95662994e118df63, scope.project.id=17423390f676473593042b205bc4d4de) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:43.219089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.220614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.229661 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 168/322] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:21:42 2022] GET /identity/v3/role_assignments?group.id=c0199018169a443f855bb26784cc7eeb&scope.project.id=17423390f676473593042b205bc4d4de&role.id=d9150fbefb504f7d95662994e118df63 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:43.230671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.236599 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.255654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.279102 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855d9ca-e2a5-47ed-a5ed-36d27e67e3a9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.321795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.322444 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.322727 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.323000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.324199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.325715 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.327425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 155/323] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:43.328362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.361061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce2648-8745-4119-b4e5-93ecece21abc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.399613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.400460 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.400989 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.401413 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.402339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:43.402794 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.404832 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.419549 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 169/324] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:43.420490 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.444467 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574b6f5-8b2a-490a-8d6f-3031d6e7d0c0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.485852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.486487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.486767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.487142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.496406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.500145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:43.505509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:43.511955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 156/325] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:43.512761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.514472 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.536321 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ee305-a5cf-4af6-9334-29549e8260ea None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.578283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.579999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.579999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.579999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.582802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.585162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.586905 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 170/326] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:43.587531 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.632496 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c6160-f501-46e9-aaed-a8d8eedec4dd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.671174 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.671745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.672023 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.672293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.696145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18199c5-58a3-42b7-bab5-d74db32553da None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.733152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.733852 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.736705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.736705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.736705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:43.736705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.736989 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.746153 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 171/327] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:43.747062 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.753021 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.755589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.757082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.761544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.851965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c908d-fe40-471d-97b3-8fb29f01ef60 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:43.859779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 157/328] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:43.864300 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.883588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:43.884188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:43.884463 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:43.884729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:43.885438 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] RBAC: Adding query filter params (group.id=c0199018169a443f855bb26784cc7eeb, role.id=d9150fbefb504f7d95662994e118df63, scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:43.885739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:43.887164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:43.892828 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 172/329] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/role_assignments?group.id=c0199018169a443f855bb26784cc7eeb&scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305&role.id=d9150fbefb504f7d95662994e118df63 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:43.893668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:43.976388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.978399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:43.981375 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c2e1a-b2e1-4d79-9768-ef15827bf118 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.003589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f622474-9417-4c13-956e-f96ec6c02546 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.018480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.019119 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.019421 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.019697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.045135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.046236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.046694 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.047239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] PATH_INFO: `/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.048519 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] RBAC: Authorizing `identity:get_project(project_id=ada30b34a9604dc8b6e53ee6f59b5305)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.050222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.052087 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 173/330] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/projects/ada30b34a9604dc8b6e53ee6f59b5305 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:44.057607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:44.095907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.097376 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.176187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cd1ec-534f-4e69-85c2-e2471f1eb52e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.217492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.219891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.219891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.219891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] PATH_INFO: `/v3/groups/c0199018169a443f855bb26784cc7eeb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.220673 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] RBAC: Authorizing `identity:get_group(group_id=c0199018169a443f855bb26784cc7eeb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.222110 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.225232 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 174/331] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:21:44 2022] GET /identity/v3/groups/c0199018169a443f855bb26784cc7eeb => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:44.227447 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:44.229888 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:44.231557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 158/332] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:21:44.243141 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:44.246188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:44.338493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379def9-7448-4404-be84-2ce39ec2110d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.338981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c09611-b638-4603-ab45-500c8e1cd322 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.370142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.370654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.370748 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.371012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.371109 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.371414 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.371633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] PATH_INFO: `/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.372030 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] RBAC: Adding query filter params (group.id=c0199018169a443f855bb26784cc7eeb, role.id=517b2b1aaac34e92b002aec2385431d0, scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:44.372340 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.372718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] RBAC: Authorizing `identity:get_project(project_id=29daf56ceab84ba1bd5c5b6f664b751b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.373765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.374129 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.375759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 159/333] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:21:44 2022] GET /identity/v3/projects/29daf56ceab84ba1bd5c5b6f664b751b => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:44.379430 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 175/334] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:21:44 2022] GET /identity/v3/role_assignments?group.id=c0199018169a443f855bb26784cc7eeb&scope.project.id=ada30b34a9604dc8b6e53ee6f59b5305&role.id=517b2b1aaac34e92b002aec2385431d0 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:44.381833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:44.455585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:44.489690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b06508-4c2c-4f14-a23f-f51d4c32050d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.525052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4f921-f4d4-4fe1-a9fb-245ae0bcd856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.525695 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4f921-f4d4-4fe1-a9fb-245ae0bcd856 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.525975 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4f921-f4d4-4fe1-a9fb-245ae0bcd856 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.526247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4f921-f4d4-4fe1-a9fb-245ae0bcd856 None admin] PATH_INFO: `/v3/groups/140f74f8a4834b04afc48782cc13ad46` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.527406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4f921-f4d4-4fe1-a9fb-245ae0bcd856 None admin] RBAC: Authorizing `identity:get_group(group_id=140f74f8a4834b04afc48782cc13ad46)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.528846 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4f921-f4d4-4fe1-a9fb-245ae0bcd856 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.530605 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 160/335] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:21:44 2022] GET /identity/v3/groups/140f74f8a4834b04afc48782cc13ad46 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:44.536830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:44.612247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:44.637307 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90dd9d-95bc-4e11-b256-48ac7de8ea6d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:44.671771 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:44.672376 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:44.672663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:44.672935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:44.673656 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] RBAC: Adding query filter params (group.id=140f74f8a4834b04afc48782cc13ad46, role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=29daf56ceab84ba1bd5c5b6f664b751b) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:44.673959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:44.675409 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42be2ba-386f-4ef3-b250-2bc8893c9c5e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:44.681799 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 176/336] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:21:44 2022] GET /identity/v3/role_assignments?group.id=140f74f8a4834b04afc48782cc13ad46&scope.project.id=29daf56ceab84ba1bd5c5b6f664b751b&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:47.171903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9902-a4e5-4c85-97f2-2303b7c54fa5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.172421 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9902-a4e5-4c85-97f2-2303b7c54fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.172421 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9902-a4e5-4c85-97f2-2303b7c54fa5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.174074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 161/337] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:47.179261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef77b1-687d-4e93-a022-5add5aa6c9aa None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.180117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef77b1-687d-4e93-a022-5add5aa6c9aa None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.180668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef77b1-687d-4e93-a022-5add5aa6c9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.204253 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cfef77b1-687d-4e93-a022-5add5aa6c9aa None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:47.205787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cfef77b1-687d-4e93-a022-5add5aa6c9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:47.284760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 177/338] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:47.480003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeef2c4-9c0e-4b0e-9ba2-17875548e315 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.480292 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeef2c4-9c0e-4b0e-9ba2-17875548e315 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.480477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeef2c4-9c0e-4b0e-9ba2-17875548e315 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.482105 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 162/339] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:47.487520 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.487808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.488087 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.516297 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:47.517499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:47.538766 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049b138-c590-4cd7-9173-aa3b967dde52 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.539044 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049b138-c590-4cd7-9173-aa3b967dde52 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.539488 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049b138-c590-4cd7-9173-aa3b967dde52 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.540907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 163/340] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:47.545742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.546025 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.546299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.568808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:47.569949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:47.620081 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:47.626716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 178/341] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:47.661928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 164/342] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:47.710795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:47.837317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fbcb7-3c57-4004-a368-3c3fddfbd2ab None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:47.876510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:47.877151 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:47.877430 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:47.877697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:47.892052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:47.893505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:47.912872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:47.913807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:47.914663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:47.914663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:47.914663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:47.916473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 179/343] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:47.923341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:47.945245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:47.991725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:48.019648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:48.025988 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36070e-c9a2-4f1d-bcee-e80053a541d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.061709 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.062247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.062526 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.062801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.063604 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:48.063794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.065135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.066920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d06d4-39d0-47dd-b61f-06d3e784f8af None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.082763 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 165/344] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:48.087821 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:48.102622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.103636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.104178 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.104666 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.116615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.118324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:48.133493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:48.134420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:48.135281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:48.135281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:48.135281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:48.137098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 180/345] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:48.138042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:48.189337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7124b-302c-4fa5-8e02-bd89f99c72d4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.218225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:48.228578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.229476 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.230002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.230416 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.242221 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.244071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.247410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddd8f0-5ca3-4258-abbf-69b6fcd62398 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.249041 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:48.261428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:48.262557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:48.263462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:48.263462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:48.263462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:48.265388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 166/346] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:48.266400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:48.288830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.289539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.289884 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.290224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.290921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.292550 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.372146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9bde6-5d40-426f-b5a8-fe3b83d040df None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.383334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 181/347] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:48 2022] POST /identity/v3/users => generated 246 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:48.383334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:48.428671 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.430337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.430623 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.430898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.431579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:48.431880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.433314 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29211875-9a0c-47e5-a0d1-795361d1bdf6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.455541 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 167/348] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:48.457191 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fec14-5d1f-4c8a-9f49-473279bbcd7c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.457482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fec14-5d1f-4c8a-9f49-473279bbcd7c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.457756 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fec14-5d1f-4c8a-9f49-473279bbcd7c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.462242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 168/349] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:48.463462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fec14-5d1f-4c8a-9f49-473279bbcd7c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:48.512509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:48.533416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1654-c5fc-4b97-8bbe-9311ed67abf2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.577875 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.578735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.579017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.579290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.579954 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:48.580255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.581626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4ebcb-9295-48dd-a396-fc5aae7120d7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.584735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fec14-5d1f-4c8a-9f49-473279bbcd7c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.596062 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 182/350] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:48.598391 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.598875 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.599359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.617653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:48.619062 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:48.622815 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd0083-2183-4325-a7a9-c5e8d1d8dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.623451 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd0083-2183-4325-a7a9-c5e8d1d8dfd2 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.623694 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd0083-2183-4325-a7a9-c5e8d1d8dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.623959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd0083-2183-4325-a7a9-c5e8d1d8dfd2 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.624604 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd0083-2183-4325-a7a9-c5e8d1d8dfd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.626028 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd0083-2183-4325-a7a9-c5e8d1d8dfd2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.633831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:48.689637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 183/351] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:48.690463 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:48.716574 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 169/352] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:48 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:48.825793 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3e3c2-ac47-4b2f-8a89-fcc4c4a38b54 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:48.880188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:48.881315 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.881596 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.881862 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.882522 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:48.883986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:48.971796 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a060fe0-3294-4672-a19a-68fcc15981d1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.972109 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a060fe0-3294-4672-a19a-68fcc15981d1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.972361 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a060fe0-3294-4672-a19a-68fcc15981d1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:48.973754 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 170/353] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:48.979475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:48.979678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:48.979938 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:49.000997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:49.002386 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:49.006964 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 184/354] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:48 2022] POST /identity/v3/users => generated 248 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:49.071207 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:49.075817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:49.084661 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 171/355] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:49.181258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f77372-7ad3-43a7-9499-10c8c5cfca2c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:49.217321 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:49.218008 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:49.218287 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:49.218558 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:49.229863 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:49.231471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:49.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:49.248996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:49.249849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:49.249849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:49.249849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:49.249849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 185/356] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:49.255134 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:49.287097 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:49.380277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0a9ca-aa17-42ee-88ea-ca5254ab36cf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:49.414646 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:49.429802 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:49.430664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:49.431155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:49.431744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:49.432339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:49.432645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:49.434216 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:49.449624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 172/357] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:49.459357 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:49.522990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:49.571644 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:49.594681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4f94c-c44a-48b6-9bb6-b98b44b4a653 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:49.608073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b190896-0e1e-4ea8-bd37-a90bca08f1cb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:49.651970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:49.652774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:49.653058 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:49.653330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:49.653964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:49.656570 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:49.673359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:49.673963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:49.674242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:49.674510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:49.675229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:49.676621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:49.730554 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.sql.core [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 05:21:49.730554 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 05:21:49.730554 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [parameters: {'id': '0b1e91b075074d788067af690dd92050', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 05:21:49.730554 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:21:49.737830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 173/358] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:49 2022] POST /identity/v3/users => generated 249 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 05:21:49.748975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:21:49.749896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application yield resource Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise self.value Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application session.commit() Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:21:49.750817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:21:49.752722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:21:49.754848 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:21:49.756853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [parameters: {'id': '0b1e91b075074d788067af690dd92050', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:49.758366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 05:21:49.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:49.770530 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 186/359] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:21:49 2022] POST /identity/v3/roles => generated 144 bytes in 338 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:49.770669 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:49.885482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba1e1e-6de8-468c-bbab-437e96d7a5fa None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:49.926993 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ca24-e200-4f94-afe1-d1e8fb532037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:49.928226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ca24-e200-4f94-afe1-d1e8fb532037 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:49.928504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ca24-e200-4f94-afe1-d1e8fb532037 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:49.928772 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ca24-e200-4f94-afe1-d1e8fb532037 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:49.946181 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:49.950859 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928ca24-e200-4f94-afe1-d1e8fb532037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1928ca24-e200-4f94-afe1-d1e8fb532037 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:49.955895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:49.958018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:49.960549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 174/360] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:21:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:49.966549 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:50.089660 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783afd2c-a28c-4466-a636-381080648a35 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:50.111462 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:50.127815 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:50.128428 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:50.128709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:50.128983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:50.129639 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:50.129944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:50.131906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f803c7-9d96-4879-8446-3d6e2d6b5def None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:50.140629 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 187/361] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:51.157630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522ac7a-ddee-44bd-b621-0d8b7dd69eec None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:51.158792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522ac7a-ddee-44bd-b621-0d8b7dd69eec None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:51.158792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522ac7a-ddee-44bd-b621-0d8b7dd69eec None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:51.159723 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 175/362] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:51.166017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:51.166782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:51.166782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:51.192797 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:51.194042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:51.292883 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 188/363] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:51.690155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522ac7a-ddee-44bd-b621-0d8b7dd69eec None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:51.817554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522ac7a-ddee-44bd-b621-0d8b7dd69eec None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:51.854088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ea020-159c-4be1-8338-5f849eb99f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:51.854732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ea020-159c-4be1-8338-5f849eb99f36 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:51.855009 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ea020-159c-4be1-8338-5f849eb99f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:51.855282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ea020-159c-4be1-8338-5f849eb99f36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:51.867482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ea020-159c-4be1-8338-5f849eb99f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-416ea020-159c-4be1-8338-5f849eb99f36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:51.873020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:51.874757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:51.876385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 176/364] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:51.881211 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:51.969624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35efcda-1982-4d64-97a6-2f31c8c16b80 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:51.969805 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35efcda-1982-4d64-97a6-2f31c8c16b80 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:51.970076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35efcda-1982-4d64-97a6-2f31c8c16b80 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:51.971807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 177/365] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:51.977958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17656b-5239-45ac-b32c-36de122999d0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:51.978663 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17656b-5239-45ac-b32c-36de122999d0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:51.979334 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17656b-5239-45ac-b32c-36de122999d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.009716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2a17656b-5239-45ac-b32c-36de122999d0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:52.011232 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2a17656b-5239-45ac-b32c-36de122999d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:52.067072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a640ae-92fd-4849-a938-b4473b2efea6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:52.081022 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:52.088868 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:52.103389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 178/366] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:52.104097 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a44b6f-38e5-4a1e-bc72-1963d1a51875 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.104566 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a44b6f-38e5-4a1e-bc72-1963d1a51875 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.105015 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a44b6f-38e5-4a1e-bc72-1963d1a51875 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.106781 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 179/367] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:52.112341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.112887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.113333 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.120220 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:52.120943 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.121306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.121584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.122243 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:52.122854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:52.125251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:52.142507 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:52.144155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:52.163822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 189/368] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:52.172891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:52.251437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 180/369] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:21:52.320080 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aec82-1c4e-4f39-ba62-4b58648e2404 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:52.353437 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:52.354053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.354338 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.354607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.364996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:52.367058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:52.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:52.382056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:52.382904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:52.382904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:52.382904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:52.382904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 190/370] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:52.388132 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:52.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9ac8-dd89-4668-bbe4-af361e7357ba None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:52.589519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:52.623858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:52.624832 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.625292 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.625698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.626666 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:52.627115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:52.629621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:52.654157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 181/371] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:52.659545 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:52.671743 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:52.721176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:52.830494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cd8d9-9382-4df9-be0e-a6f7754b129a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:52.855468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261af5e1-55f0-46e3-b85c-00a869571ce5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:52.868097 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:52.868808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.869146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.869526 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.880284 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:52.887330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:52.888340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:52.889295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 182/372] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:52.889954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:52.920193 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:52.920825 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:52.921101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:52.921371 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:52.929087 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:52.930534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:21:52.944216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:52.946488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 191/373] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:21:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:52.947191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:52.964461 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.014054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1774-af6b-4950-b740-a26346f9cc6a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.059596 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.065317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.066474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.067164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.067615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.074887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8dc16-ca73-49e9-9bf9-0a3a55084009 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.082342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:53.089384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:53.090291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:53.091101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:53.092407 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 183/374] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:53.093049 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.120477 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.121443 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.122117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.122545 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.123921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:53.124487 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.126650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.138145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 192/375] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:53.139164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.220744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.224767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b207-f56b-40d6-abce-287b26935e83 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.261736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.262526 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.262920 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.263265 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.264332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:53.264491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.266008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.270703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb3be1-433e-41c7-824b-daf591aa33e5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.305878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 184/376] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:53.306901 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.315431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.316069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.316349 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.316619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.317488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:53.317593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.318973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fba47-7ae5-462a-8529-b14cde8a4867 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.327118 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 193/377] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:53.328390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab3fe-6b66-4c10-9447-62039866015d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.328674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab3fe-6b66-4c10-9447-62039866015d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.328951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab3fe-6b66-4c10-9447-62039866015d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.331327 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 194/378] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:53.331327 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deab3fe-6b66-4c10-9447-62039866015d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.343537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.412388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302ff11-1cd9-4946-b514-b0cfbec269ed None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.430548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deab3fe-6b66-4c10-9447-62039866015d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.446197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.446900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.447267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.447578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.458281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.460412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.463603 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.464227 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.464508 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.464788 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.474483 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-19f84791-e90b-4f3a-929c-b34acb1190ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:53.474789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:53.475883 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:53.476716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:53.476716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:53.476716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:53.476880 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.480550 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 185/379] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:53.481767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd449-8ec9-41de-aad6-fb99856bc6f8 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.482046 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd449-8ec9-41de-aad6-fb99856bc6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.482315 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd449-8ec9-41de-aad6-fb99856bc6f8 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.483566 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 186/380] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:53.484244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd449-8ec9-41de-aad6-fb99856bc6f8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-38b21c1d-7ca2-4e3d-8458-e320d1e95364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:53.495586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:53.496505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:53.497496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:53.497496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:53.497496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:53.497496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 195/381] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:53.498057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.498166 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.498241 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.498368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.516523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:53.517718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:53.572859 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.582666 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 196/382] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:53.586090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.587250 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd449-8ec9-41de-aad6-fb99856bc6f8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.623437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.623991 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.624271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.624539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.636329 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.637936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-72df5c6f-684c-4585-8cca-e91dcf78f942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:53.652708 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:53.653691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:53.654536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:53.654536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:53.654536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:53.654536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 187/383] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:53.655326 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.655326 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.655626 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.675278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:53.676474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:53.690988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49912b23-ec8b-4ced-a7e1-99209a38d38a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.725782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.726392 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.726677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.726955 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.727618 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:53.727922 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.729288 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.736499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.746001 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 197/384] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:53.746001 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.751887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 188/385] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:53.757745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.826220 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:53.877770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60198f5-abc1-4b91-a179-9a32acf66544 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.907767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe708d2-9e82-4cc7-98c9-09f0ce9a16a6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:53.912267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.912866 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.913152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.913427 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.914046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:53.914344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.915751 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.930052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 198/386] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:53.930870 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:53.949367 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:53.949976 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:53.950254 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:53.950524 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:53.951155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:53.951485 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:53.952850 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:53.965592 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 189/387] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:53.966434 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.018814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.042500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edcea4-b1b0-4a1a-aec4-c1777d078ca1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.055866 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.067183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dd8b8-3be5-4e9a-9675-8e34ac294d49 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.075482 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.076106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.076382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.076651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] PATH_INFO: `/v3/users/glance` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.083427 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.085809 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:21:54.097176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:54.098100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 199/388] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:54.099976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.102425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.103053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.103597 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.103867 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.117088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.118515 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:54.131982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:54.132922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:54.133816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:54.133816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:54.133816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:54.135583 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 190/389] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:54.136343 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.197981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30b473-ae74-4b4a-b549-df0896be52db None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.230221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.230833 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.231110 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.231410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.232070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25513e7-7ddb-4548-92da-405ce183f63a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.234251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.237703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.239150 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:21:54.251615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:54.252536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:54.253394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 200/390] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:54.253886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.269104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.269721 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.269997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.270268 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.277623 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.279540 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.284061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:21:54.292538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:54.293649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:54.294536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 191/391] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:54.295155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.359803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd964-d7c9-4f5f-8c4b-d0ed2cecb7f5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.393414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.393964 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.394242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.394510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.395919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c96f0-5fb9-4ef3-afb2-d9f179f3bc5d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.404728 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:54.409844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:54.411100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 201/392] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:54.412159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.438320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.438922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.439199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.439502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.440222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:54.440528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.441949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.459145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 192/393] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:54.459967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.482606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.506082 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.514383 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae16d-62aa-4196-927b-38091d157fac None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.547641 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.548234 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.548506 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.548771 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.549393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:54.549691 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.551058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.559072 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd628159-b256-466d-922f-a6223f323cf8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.561952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 202/394] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:54.562933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.594953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.595572 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.595856 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.596124 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.596800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:54.597104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.598524 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.615336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 193/395] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:54.616063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.662390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18e36-ebef-4e45-bf6a-ba3de41670e8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.694253 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.694845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.695117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.695419 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.696124 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:54.696423 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.697901 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.698012 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.715081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522c6d4-3622-4140-8a5f-67bba19ae8f3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.733470 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 203/396] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:54.736609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.746657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.751861 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.752463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.752739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.753008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.753658 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:54.753956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.755315 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.763111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 194/397] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:54.763962 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.833507 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f682de-0e56-4de1-85f8-8906ff3e9fa7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.860717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea4071-178b-46be-b0ae-4d07a8cd6a9c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:54.866831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.867483 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.867767 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.868036 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.877614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.879061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.892943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:54.893101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:54.894003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:54.894858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:54.894858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:54.894858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:54.895024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:54.895024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:54.895024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:54.895024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:54.895742 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 204/398] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:54.896121 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:54.896682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.954911 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.965753 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:54.970261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 195/399] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:21:54 2022] POST /identity/v3/users => generated 247 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:21:54.971040 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:54.996104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006e24d-2bdb-416d-af50-6d5d4a71cfbe None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.034659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.035368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.035909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.036184 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/231350e78b2740a4b0aef761f7baa4e6/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.054157 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=231350e78b2740a4b0aef761f7baa4e6, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.056368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.082568 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501117-955b-4500-903e-bc1dc0595412 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.119693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 205/400] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:54 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/231350e78b2740a4b0aef761f7baa4e6/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:55.120740 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.122925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.123633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.123972 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.124248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.136848 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.139004 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:55.156408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:55.157418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:55.158269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:55.158269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:55.158269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:55.162624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 196/401] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:55.163607 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.235215 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:55.289931 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4f05f-32d7-4050-948d-e8319976dc26 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.294893 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269d59-08f9-4082-a644-686ee42b022a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.305734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:55.332056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.333295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.333295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.333295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.344579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.344889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.345617 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.346088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.346420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.346661 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.347510 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:55.347955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.349999 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.362712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 197/402] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:55.363746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:55.364900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:55.365963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:55.365963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:55.365963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:55.365963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 206/403] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:55.370534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.370813 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.479246 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0cfef-a416-4b52-9932-764fe72e0c57 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.491508 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db7921-4da4-428d-9a41-1f38a82ff4ec None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.507251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:55.516875 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.519362 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.519362 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.519362 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.525725 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.527348 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.529643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.530830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.531360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.531954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.532679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:55.533059 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.534494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:21:55.541579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:55.542602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:55.543909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:55.544879 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 207/404] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:55.549499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.553192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 198/405] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:55.554074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.575724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:55.651465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27d630-61d3-43b2-ac3b-dcf04ee8d6df None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.668309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60de2c5-b815-4e47-8a85-6952103dc5e6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.688832 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.689439 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.689720 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.689988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.690731 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:55.691077 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.692595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.705191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 208/406] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:55.706080 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.711779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.712403 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.712685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.712973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.713670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:55.713975 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.715524 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.729602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 199/407] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:55.730595 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.763875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:55.819467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:55.839545 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd1d6d-23ec-452c-90ab-f7e6bdae47d4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.841111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf10ad0-944a-443e-ac0e-ff077a3722a6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:55.876796 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.877449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.877949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.878223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.884196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:55.884919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:55.885270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:55.885874 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:55.886360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.888032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.893619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:55.895498 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:21:55.901184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:55.902710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 200/408] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:55.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:21:55.915627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:55.917215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:55.918873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 209/409] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:55.919518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.014774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:56.026225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225a8e-16bb-4bb8-a7ef-136835dd99ff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.056791 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0eca-2898-4f38-a9f1-e0ab65c22d7c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.070435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.071686 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.072170 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.072644 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/7b0074deddf841749794e5cba04ed3d3/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.089356 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=7b0074deddf841749794e5cba04ed3d3, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.091028 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.096598 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.097229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.097515 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.097791 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.098508 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:56.098817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.100819 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.118218 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 210/410] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Oct 17 05:21:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:56.119477 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.124920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:56.151499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 201/411] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:55 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/7b0074deddf841749794e5cba04ed3d3/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:56.153512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.283892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85662f1-6a90-4cc1-8fe0-5c43f1048ed2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.305488 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2931d4-f86d-4de2-b261-66a96f9ee26d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.338099 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.339534 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.341397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.341890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.362123 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.364459 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.367523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:56.379820 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.380483 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.384105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.384105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.384105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:56.384105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.384105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:56.402118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:56.403876 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:56.404943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:56.404943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:56.404943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:56.404943 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 202/412] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:56.407539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 211/413] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:56.413750 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.421467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.504651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:56.557130 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a68c07-0393-4bd0-a639-6697107f57a3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.577161 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2c4c-c113-4c58-8d45-c6289596b91f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.624595 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.625214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.625489 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.625757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.626388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:56.626684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.628069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.636058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.636830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.637117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.637394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/f182987c54cc449296ef3d736e471127/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.640560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 203/414] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:56.652395 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.654613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=f182987c54cc449296ef3d736e471127, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.656184 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.715043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:56.743028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 212/415] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:56 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/f182987c54cc449296ef3d736e471127/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:56.773840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37d463-8d92-4e40-9150-850436f21f31 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:56.813261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:56.814134 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:56.814573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:56.815047 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:56.823938 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:56.825632 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:21:56.839381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:56.840474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:21:56.841533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:56.843668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 204/416] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:56.849361 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:56.986166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f4695-db4d-4e15-a142-d19a616f6bbb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:57.000117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:57.022623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:57.023238 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:57.023588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:57.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:57.024598 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:57.024907 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:57.026412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d1913-573f-44b4-a7bd-d63f9f587c20 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:57.039792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 213/417] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:57.045460 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:57.156184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a690d22-cd0c-4401-8d42-a7f44c4e3201 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:57.177226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:57.197355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22790f5-60b2-4557-80d5-cb3d13c0b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:57.198017 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22790f5-60b2-4557-80d5-cb3d13c0b2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:57.198295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22790f5-60b2-4557-80d5-cb3d13c0b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:57.198564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22790f5-60b2-4557-80d5-cb3d13c0b2f8 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/937548ffd2d040f6a707f515faa6a4e7/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:57.214690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22790f5-60b2-4557-80d5-cb3d13c0b2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=937548ffd2d040f6a707f515faa6a4e7, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:57.216152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22790f5-60b2-4557-80d5-cb3d13c0b2f8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:57.303507 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 205/418] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:21:57 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/937548ffd2d040f6a707f515faa6a4e7/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:21:57.425884 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b6fc-83ba-478e-8fcb-f37bb2e17c1e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:57.426416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b6fc-83ba-478e-8fcb-f37bb2e17c1e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:57.426416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b6fc-83ba-478e-8fcb-f37bb2e17c1e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:57.428317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 214/419] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:57.433678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:57.434186 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:57.434288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:57.460316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:57.461562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:57.530681 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefc339-24f4-49e5-8d9a-a53edcb863c7 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:57.531004 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefc339-24f4-49e5-8d9a-a53edcb863c7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:57.531233 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefc339-24f4-49e5-8d9a-a53edcb863c7 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:57.532729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 215/420] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:21:57.538963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:57.539539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:57.540122 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:57.559378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:57.561116 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:57.567276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 206/421] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:57.637250 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 216/422] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:57.906153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:57.979349 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.011289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:58.018673 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384625e-ffbd-4e7d-a80b-4615cf6253a8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.073269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.073903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.074181 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.074453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.096903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:58.101767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:58.102667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:58.103511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 207/423] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:58.111808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.184699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779ca6-8231-4dc6-8f6d-85bb9b2e1263 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.186845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:58.249724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.250356 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.250633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.250902 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.272219 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:58.276981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:58.277839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:58.278642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 217/424] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:58.289026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.291414 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae270f-c6e5-40ef-bc87-1dea6199f22b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.329286 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.329902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.330180 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.330450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.331111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:58.331457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:58.333201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:58.346173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 208/425] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:58.352097 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.386259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:58.495613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304c345-ebae-448c-a033-0dd0836af265 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.541062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191140e0-72bf-46bf-ab81-637e8a025158 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.571387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.572004 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.572283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.572557 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.573217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:58.573521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:58.574902 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:58.587237 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 218/426] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:58.592454 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.609281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.610101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.610517 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.610924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.619469 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:58.633885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:58.635348 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:58.662288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:58.663479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:58.664544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:58.664544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:58.664544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:58.671490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 209/427] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:58.677313 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.700933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c4ae0-bdb5-447a-ab04-6875a4c5e656 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.736854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.747544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.747544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.747544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.751356 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:58.753197 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:58.771626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:58.772632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:58.773484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:58.773484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:58.773484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:58.775555 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 219/428] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:21:58.785002 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.810253 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:58.871353 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fede21-0176-4637-8bbf-e6a5c0b9234a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.918295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:58.919927 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.920551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.920816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.921128 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.921809 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:58.922108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:58.923655 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755b65-f322-469f-8ace-41ac558e9ea3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:58.937919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 210/429] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:58.939352 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4276-2db3-4ae7-972e-dffeb5ab70c3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.939689 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4276-2db3-4ae7-972e-dffeb5ab70c3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.939982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c4276-2db3-4ae7-972e-dffeb5ab70c3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.942273 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 211/430] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:58.943852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4276-2db3-4ae7-972e-dffeb5ab70c3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:58.952225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ae5e-62a8-41c0-9513-b8def0d1af97 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:58.996337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:58.997380 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:58.998024 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:58.998250 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:58.998941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:58.999252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.001464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b12b31-05f6-4561-8fe1-26022602394c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.020833 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 220/431] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:59.024425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.024712 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.024986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.044153 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:59.045336 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:59.069442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c4276-2db3-4ae7-972e-dffeb5ab70c3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.098741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:59.107910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.108724 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.109149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.109569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.123351 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 221/432] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:59.123351 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.125244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.126865 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:21:59.144448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:59.145602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:21:59.147171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:59.155471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 212/433] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:21:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:59.160389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.189500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:59.226357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c557b3-08fd-4fdd-bd1f-1e0db22899c3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.260529 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.261734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.262017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.262291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.270032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.271562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:21:59.283511 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:59.284410 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:59.285328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 222/434] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:59.296663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.387829 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f3764-ec86-48a5-a9f5-e5e0f24ded02 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.449369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.449983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.450261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.450528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.451379 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:59.451685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.453214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.467819 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d5bd4-5180-42b2-bc4c-8af5ef213f85 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.477578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 213/435] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:59.478487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.495261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:59.504543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.505350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.505692 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.506100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.506880 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:59.507247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.508959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.528449 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 223/436] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:21:59.531435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.532161 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:59.587472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a887c-5e86-4201-a58e-e4fab0f73d69 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.623635 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc40f4d-2e3c-4dcf-916a-e1bce70212a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.624498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc40f4d-2e3c-4dcf-916a-e1bce70212a6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.624973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc40f4d-2e3c-4dcf-916a-e1bce70212a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.625249 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc40f4d-2e3c-4dcf-916a-e1bce70212a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.637605 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc40f4d-2e3c-4dcf-916a-e1bce70212a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:21:59.639211 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99775148-7a05-4ae8-8c62-d30c4aa7d9c3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5dc40f4d-2e3c-4dcf-916a-e1bce70212a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:21:59.643543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:59.644410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:21:59.645203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:21:59.647119 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 214/437] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:59.648542 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b33b2-696e-4723-838d-304d4fe805ba None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.648829 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b33b2-696e-4723-838d-304d4fe805ba None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.649103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b33b2-696e-4723-838d-304d4fe805ba None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.650426 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 215/438] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:21:59.651238 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b33b2-696e-4723-838d-304d4fe805ba None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.682185 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.683799 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.684291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.684792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.697826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.700118 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:21:59.717783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:21:59.718743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:21:59.720320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:21:59.720320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:21:59.720320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:21:59.720320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 224/439] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:21:59.720320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.761413 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b33b2-696e-4723-838d-304d4fe805ba None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.765736 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:59.798530 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:21:59.827575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.828229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.828517 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.828786 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.837284 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdfcc-c25c-4852-9abc-795c8806c7b3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:21:59.872626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:21:59.873285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.873566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.873836 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.874545 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:21:59.874845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.876226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b32f2-2afb-49d8-8f3a-55e9e7f3d8e1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.884649 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 225/440] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:21:59.885829 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:21:59.886111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:21:59.886382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:21:59.903523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:21:59.904965 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:21:59.905070 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:21:59.906180 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:21:59.969137 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 226/441] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:21:59.969946 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:21:59.997826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.012294 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 216/442] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:22:00.017764 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.025698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.074423 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd3fc-23ff-4f20-9a8a-bca18776c64e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.108056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.108653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.108929 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.109195 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.109817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:00.110114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.111501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b229a0-ffe1-433e-b509-b8b9dbaae832 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.127612 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 227/443] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:00.127769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a8674-7426-4a78-855d-511c5cd51421 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.127887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388ab3-d0be-47fb-89b6-4526d790ea07 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.127887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388ab3-d0be-47fb-89b6-4526d790ea07 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.127887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388ab3-d0be-47fb-89b6-4526d790ea07 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.127887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 228/444] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:00.127887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388ab3-d0be-47fb-89b6-4526d790ea07 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.164554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.165187 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.165466 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.165738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.175846 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.177315 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d8ff1cfd-3062-44e1-b9c2-2be2c113e318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:00.195989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:00.197063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:00.197915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:00.197915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:00.197915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:00.197915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 217/445] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:00.198218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.198423 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.198693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.218448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:00.220303 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:00.223331 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.238528 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388ab3-d0be-47fb-89b6-4526d790ea07 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.273334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.273665 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.273953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.274232 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.274564 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.284356 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.286066 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.287208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 218/446] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:00.288009 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:00.319934 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:00.320856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:00.321717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:00.321717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:00.321717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:00.323642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 229/447] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:00.324655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.392658 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc3464-4b4e-4327-b7ac-d373f1034e19 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.429503 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.430152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.430425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.430694 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.434013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c758ca-d670-402c-80df-5cd2e860d692 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.437377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.438800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:00.450192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:00.451053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 219/448] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:00.452270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.475829 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.476436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.476717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.476991 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.477618 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:00.477930 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.479314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.489556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.498147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 230/449] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:00.498147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.507074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.563199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ae5dc-8a2e-4100-aeaf-5c8cdaee6002 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.596941 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.597552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.597552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.597834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.599489 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1691c-dfe4-4f12-9033-82d534664dbd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.607551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:00.612702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:00.613690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:00.614479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 220/450] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:00.616263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.634589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.635193 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.635490 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.635762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.636388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:00.636693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.638074 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.649150 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 231/451] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:00.649921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.706082 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.714535 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ef228-55ef-4ffb-adb3-e5dbd2137ed2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.723428 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.746173 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa55a9b-30b5-4734-b734-24f47aede4f8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.748015 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.748622 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.748899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.749171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.749852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:00.750157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.751654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.764406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 221/452] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:00.765110 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.783911 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.784539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.785233 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.785515 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.793735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.795166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:00.806058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:00.807115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 232/453] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:00.808060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.878560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8302e9-47d9-4a72-b449-92a88292cbfe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.903972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa1756-c812-48bc-885d-c6e633d3b242 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:00.911771 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.912432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.912717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.912982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.922944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:00.924618 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:00.928246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:00.929304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:00.930293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 222/454] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:00.931097 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.941776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:00.942379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:00.945103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:00.945103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:00.945103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:00.945103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:00.945859 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:00.954727 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 233/455] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:00.955502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:00.962595 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.033912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf684-9a81-4422-88db-5f56ed3cdff0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.051961 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0cae8-2adc-4bc2-947f-4ac74b289117 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.066526 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.067146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.067480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.068302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.074729 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.076234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.085700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.086334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.086608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.086876 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/b04f725dd19b4a0f9669840fba801b3f/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:01.087707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:01.088677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 223/456] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:01.094495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.107414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=b04f725dd19b4a0f9669840fba801b3f, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.108866 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.173082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.175162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 234/457] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:00 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/b04f725dd19b4a0f9669840fba801b3f/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:22:01.176003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.187483 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2e57-997e-4675-becc-50f52acd9be9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.219216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.225848 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.226439 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.226720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.226993 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.227766 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:01.228078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.229555 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.251216 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 224/458] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:01.252119 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.307020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dcca9-5618-49de-a912-34b0b7850203 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.345234 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.345849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.346126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.346394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.347055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:01.347389 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.348756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.359251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 235/459] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:01.360357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.370548 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23aa62-fa83-4012-b60e-654c2cf1489c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.405458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.406348 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.406624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.406892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.420069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.421563 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.429528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:01.437777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:01.438869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:01.440260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:01.440260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:01.440260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:01.440260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 225/460] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:01.440260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.470256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.477585 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9734e6-fedf-40ac-940a-fcd9e10de655 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.513945 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.514549 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.514824 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.515091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.515797 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:01.516099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.517636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.535006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 236/461] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:01.535006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.548987 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1175-407f-43c3-a5df-ff7087c719fd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.584766 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.585374 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.585648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.585919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.634729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032dba-2e58-4cfe-8558-42540eb6f119 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.655558 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.659821 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.662942 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.669697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.670323 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.672250 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.672250 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.681384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.682791 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:01.697398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:01.698502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:01.699682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:01.699682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:01.699682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:01.699682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 237/462] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:01.699682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.710178 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.781620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 226/463] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:22:01.781620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.810050 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab934069-aae8-44eb-8927-09c7d7fddcab None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.844858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.845465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.845746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.846022 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.846656 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:01.846957 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.848372 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.865493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 238/464] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:01.865493 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.886996 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.891270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458724f-c573-48b5-bffa-35ece80ef1dc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:01.928693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:01.929302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:01.929603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:01.930061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:01.930888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:01.932077 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=231350e78b2740a4b0aef761f7baa4e6) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:01.932342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:01.934176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:01.951505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 227/465] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/role_assignments?user.id=231350e78b2740a4b0aef761f7baa4e6&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:01.951505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:01.985271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec051e-b76b-432f-8f76-ece855ec3139 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.039319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.040962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.041242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.041570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.042241 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:02.042542 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.043991 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.067611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 239/466] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:02.070647 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.081854 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6bd15-be6a-45cc-99fb-a93489c0f6a9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.122561 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.122561 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.122990 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.123383 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.134776 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.136453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:02.153773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:02.154857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:02.155860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:02.155860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:02.155860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:02.155860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 228/467] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:02.156252 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.177755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.210102 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e84b31-5df2-466f-ad8c-122a1e473aed None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.243533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.253158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.254047 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.254491 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.255946 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.262705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.264456 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:02.277676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:02.278613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:22:02.279546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:02.280048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badf29c-7483-440a-99c1-2fd87c5ddc27 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.287358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 240/468] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:02.288464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.320502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.321180 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.321453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.321721 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.330187 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.332236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:02.347164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:02.348263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 229/469] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:02.349271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.420442 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930c1f8-daf2-4b08-9879-31106d01bf9f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.454629 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.468668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.468668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.468668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.468668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.469046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:02.469046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.469046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.484323 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 241/470] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:02.485174 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.486302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90798cc-494e-4843-82fe-2cac7aed8f44 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.534851 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.535498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.536038 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.536316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.537184 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:02.537553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.539146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.541753 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.560457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 230/471] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:02.561304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.611348 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e9207-3ee2-496c-9cee-29595abec801 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.647105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.648510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.648939 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.649357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.650332 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:02.650782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.652431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.666674 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4552f-845a-4b9a-8a05-e41f6b3307d7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.672308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 242/472] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:02.673451 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.697666 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.702838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.703505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.703792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.704135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.712397 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.714657 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:02.728911 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:02.729958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:02.730898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:02.737777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 231/473] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:02.737777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:02.801425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64bd31-111b-461e-9c17-b4a8a351fcbc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.805747 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.841166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.841879 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.842223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.842505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.860578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405b07-596b-4a37-a1f5-0946441362ba None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:02.899936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:02.900838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:02.901265 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:02.901717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:02.917257 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.918935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:02.980797 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:02.993387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:02.995475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.028060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 243/474] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:22:03.028996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.032363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.116694 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 232/475] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:22:03.118176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.136835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea285a-6d55-4fac-9238-e599c23527e3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.172688 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.173302 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.173587 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.173864 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.174548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:03.174858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.176333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.194094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 244/476] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:03.195026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.239208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.252391 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f85b75-7d83-47e2-8911-c550010d5069 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.266581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.304624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.305591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.305878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.306154 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.313932 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ddfc1-5fe7-4d9c-87b1-54aeec28991e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.319784 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.321244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:03.341316 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:03.342751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:03.343978 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:03.343978 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:03.343978 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:03.343978 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 233/477] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:03.343978 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.356185 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.357416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.357923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.358457 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.371033 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.372736 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:03.388012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:03.389100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:03.389990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:03.389990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:03.389990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:03.389990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 245/478] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:03.389990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.460343 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6870df2-5a46-4abc-b767-fcd04a6204a4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.495812 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.496444 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.496620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.496902 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.497627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=7b0074deddf841749794e5cba04ed3d3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:03.497940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.499369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e3054-6be8-4336-97a5-f1bac5db2ab6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.511475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 234/479] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/role_assignments?user.id=7b0074deddf841749794e5cba04ed3d3&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:03.511475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a673956-dbf1-4736-8769-aaca91d9a3e7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.511749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bccbd-134c-43ab-93e7-179bb24f577e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.511834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a673956-dbf1-4736-8769-aaca91d9a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.511834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a673956-dbf1-4736-8769-aaca91d9a3e7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.511834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 235/480] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:03.511834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a673956-dbf1-4736-8769-aaca91d9a3e7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.525400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.530750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.555971 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.556741 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.557122 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.557516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.559684 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:03.560104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.561756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26882f5-c6f8-43fc-92f3-acf1a838a21a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.579558 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 246/481] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:03.581169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.581573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.581941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.602724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:03.604217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:03.671362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a673956-dbf1-4736-8769-aaca91d9a3e7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.693501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 247/482] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:03.695400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.723382 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.724825 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.725475 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.726479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.727567 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:03.727906 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.729383 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.743200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 236/483] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:03.748483 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.821929 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e6908-d9bc-48d0-977a-b099f9688ed8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.830643 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.839652 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:03.867942 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.869049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.869339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.869621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.879106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.880942 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.882024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45185eb3-02a2-42ee-b028-fcd63f64a5ef None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:03.896998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:03.897949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:03.898792 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 248/484] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:03.906609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:03.924085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:03.924729 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:03.925018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:03.925302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:03.935579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:03.937050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:03.952793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:03.953670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:03.954522 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 237/485] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:03.959602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.041365 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100bcc-9728-429e-91b0-e74b77a96b1d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.078533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.079336 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.079697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.080044 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.080796 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:04.081196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.082999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.092370 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf128a7-f2a9-4145-88b0-e3a13145a3eb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.102643 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 249/486] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:04.102643 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.133404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:04.136805 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:04.146451 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.151220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.151220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.151220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.151220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:04.151220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.151220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.168495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 238/487] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:04.171083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.227331 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa0718-7918-405c-9888-767c9fa16248 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.264380 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.265072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.265834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.266112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.278169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:04.282933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:04.283855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:04.284650 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 250/488] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:04.285311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.303657 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abea15-c6e0-48c6-83ab-291e909980d7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.339137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.339732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.340017 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.340286 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.341008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=f182987c54cc449296ef3d736e471127) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:04.341637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.342742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fcfa2-6a5a-4798-8889-2e3083cccb69 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.348743 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 239/489] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/role_assignments?user.id=f182987c54cc449296ef3d736e471127&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:04.350453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0876-20c2-4677-84f7-eb7fbe0832b9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.350744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0876-20c2-4677-84f7-eb7fbe0832b9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.351019 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0876-20c2-4677-84f7-eb7fbe0832b9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.356365 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 240/490] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:04.356365 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0876-20c2-4677-84f7-eb7fbe0832b9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.392872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:04.410311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c49b4-3ad6-4b7d-bffa-647ccc5836a7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.439889 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:04.453639 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.454264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.454581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.454924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.455872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=937548ffd2d040f6a707f515faa6a4e7) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:04.456057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.457604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146765f0-d7aa-47da-ab44-4d703fd57de3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.470229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 251/491] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/role_assignments?user.id=937548ffd2d040f6a707f515faa6a4e7&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:04.470229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.470229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.470229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.499267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:04.501896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:04.549218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0876-20c2-4677-84f7-eb7fbe0832b9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.590442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.591070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.591374 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.591651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.592334 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:04.592636 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.594177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.603717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 241/492] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:04.612346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.659399 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 252/493] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:04.742710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47202a5-b33b-4e78-979e-6311e75780f6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.757443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:04.779015 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.779690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.780095 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.780336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.793034 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.794590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:04.809163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:04.810290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:04.811140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:04.811140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:04.811140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:04.814938 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 242/494] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:04.816374 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.898480 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:04.918243 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd80dea-a38c-4a2b-b907-8f316749abb3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:04.955692 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:04.956308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:04.956590 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:04.956863 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:04.957510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:04.957807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:04.959252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:04.974922 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 253/495] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:04.977584 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:04.998409 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.106212 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffcf26-7414-4bff-9d82-c67bea679876 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.141687 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.142321 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.142604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.142878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.157296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9207-6b56-46c4-84ba-baa9840a8bdc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.161939 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:05.168068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:05.169154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:05.169973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:05.175329 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:05.179765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 254/496] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:05.182680 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.231384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.232021 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.232297 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.232568 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.247627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:05.249071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:05.268327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:05.269319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:22:05.270295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:05.282773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 243/497] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:05.283679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.325390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:05.343541 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bc87-728c-4b19-a774-32d09e779c87 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.378495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.379452 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.379886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.380389 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.381288 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:05.381732 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:05.383326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:05.396981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 255/498] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:05.398432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.471690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:05.511569 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1eb6c-39fa-4352-9ea1-aaf350abf6c7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.518037 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beacba-4f4a-41a3-b01a-846d8394d12c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.549747 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.550653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.551078 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.551699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.563613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:05.565266 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:05.581699 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:05.582839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:05.585026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:05.585026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:05.585026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:05.591254 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 256/499] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:05.592628 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.656059 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.656638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.656924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.657205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.657940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:05.658247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:05.659717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:05.705188 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 244/500] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:22:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:05.711488 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.714282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d75c65-42f6-4a36-a23e-1ff3032efb8e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.747680 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:05.796788 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.797480 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.797770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.798152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.798818 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:05.799121 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:05.800573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:05.809250 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:05.822277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed79210-df02-4377-8afa-7a2bb91e623c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:05.834279 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 257/501] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:05.852263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:05.863674 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:05.864284 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:05.865100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:05.865369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:05.939926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:05.941429 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.030563 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:06.054274 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 245/502] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:22:06.056400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.061468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a65802-9216-41de-afcf-4694f8aff065 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.110697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.111348 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.111633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.111894 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.119620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.121058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:06.132860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:06.134787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:06.135965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 258/503] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:22:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:06.152306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.167108 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c160-ac5f-4988-a8dd-9243cd677a40 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.210349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.210969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.211245 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.211541 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.223126 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.224569 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:06.225973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-787d3436-a4c2-4399-95e0-0d7cae7faec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:06.242449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:06.244083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:06.244982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:06.244982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:06.244982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:06.248214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 246/504] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:06.249476 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70553b-c9f4-4744-8e44-7c53d4a46cd0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.249759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70553b-c9f4-4744-8e44-7c53d4a46cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.250033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70553b-c9f4-4744-8e44-7c53d4a46cd0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.255340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 247/505] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:06.255340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70553b-c9f4-4744-8e44-7c53d4a46cd0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.299404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:06.308402 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1970-95bf-48ff-bab2-87625c438b76 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.344558 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.345523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.346010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.346486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.347432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:06.348034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.350346 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5e186-be2e-47a1-bfa3-9aeaf6962697 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.369207 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 259/506] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:06.370709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.371204 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.371765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.387692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70553b-c9f4-4744-8e44-7c53d4a46cd0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.390390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:06.391798 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:06.429674 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.430498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.430924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.431607 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.432850 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:06.433578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.435160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.453052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 248/507] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:06.453052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.480471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 260/508] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:06.481801 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.527263 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:06.611934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f792767-11c7-488c-8627-b175a7821118 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.615613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:06.683329 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.683977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.684261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.684536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.685633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53ff7-f799-4f88-b06e-064d84408a4d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.708851 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.710305 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:06.741519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:06.742667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:06.743728 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:06.743728 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:06.743728 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:06.743896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.743985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 249/509] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:06.744064 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.744064 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.744064 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.750316 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.750591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.751942 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:06.765534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:06.766499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:06.767434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 261/510] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:06.774379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.881923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:06.893584 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e60b3-d05d-4044-a45f-a4b01131388d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.904738 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f556089-87ca-4529-9586-ffbe9215a93d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:06.931164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.932201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.932684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.933158 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.934106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:06.934680 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.936369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.947636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:06.948526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:06.948810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:06.949088 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:06.949772 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:06.950077 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:06.955360 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:06.955935 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 250/511] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:06.956825 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.964566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 262/512] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:06.965820 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:06.970201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.063957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aca3b1-8d0d-4e75-83eb-6c807452eea7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.075865 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073cfad-0c1d-45a4-83fd-82ea7396a5fa None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.102808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.103480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.103794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.104038 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.111464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.112450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.112744 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.113018 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.114073 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.115574 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.119867 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.121294 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-bfd2593f-7605-468d-81e8-332f559f2f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:07.130641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:07.132150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:07.133076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:07.133076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:07.133076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:07.133076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 251/513] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:07.133076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f15048-71d9-43bc-85c9-13a9197ea1d2 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.133452 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f15048-71d9-43bc-85c9-13a9197ea1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.133520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f15048-71d9-43bc-85c9-13a9197ea1d2 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.134788 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 252/514] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:07.136359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:07.137215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:07.138099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:07.139725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f15048-71d9-43bc-85c9-13a9197ea1d2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.140244 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 263/515] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:07.141189 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.144073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.207169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.241176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773547c-bbc0-40a3-ad3c-2e3352f1dd93 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.249531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f15048-71d9-43bc-85c9-13a9197ea1d2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.274361 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.274955 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.275229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.275524 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.276143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:07.276439 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.277804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ad303-05d9-4436-889c-bb820ca7050c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.283838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.284432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.284712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.284986 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.285951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=b04f725dd19b4a0f9669840fba801b3f) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:07.286292 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.287756 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.289569 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 264/516] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:07.290723 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce2d2c-f00d-43e9-8c72-09f375f79ada None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.291004 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce2d2c-f00d-43e9-8c72-09f375f79ada None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.291274 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce2d2c-f00d-43e9-8c72-09f375f79ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.301863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 253/517] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:07 2022] GET /identity/v3/role_assignments?user.id=b04f725dd19b4a0f9669840fba801b3f&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:07.301863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.315822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-88ce2d2c-f00d-43e9-8c72-09f375f79ada None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:07.316885 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-88ce2d2c-f00d-43e9-8c72-09f375f79ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:07.398663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.406793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830291fb-63f2-4867-a850-c7d417a4831f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.427326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 265/518] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:07.427326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e14d05-b349-4c7e-83fb-2c655f0777a1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.427326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e14d05-b349-4c7e-83fb-2c655f0777a1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.427326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e14d05-b349-4c7e-83fb-2c655f0777a1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.427326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 266/519] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:07.427326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e14d05-b349-4c7e-83fb-2c655f0777a1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.446151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.457513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.458937 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.459282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.459818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.460587 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:07.460964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.463224 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfd45d-1ad1-4084-89b8-54353623d295 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.478480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 254/520] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:07.480145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.480434 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.480706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.502809 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:07.504193 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:07.574937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e14d05-b349-4c7e-83fb-2c655f0777a1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.595395 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 255/521] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:07.596251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.649336 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.649859 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.650136 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.650407 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.651041 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.652494 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.725321 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe5c31-20e2-45ae-9d68-80516edfd5df None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.739400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.765894 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 267/522] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:22:07 2022] POST /identity/v3/users => generated 254 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:07.765894 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.770204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.770920 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.771197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.771494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/231350e78b2740a4b0aef761f7baa4e6/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.774318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=231350e78b2740a4b0aef761f7baa4e6, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.775806 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.833397 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:07.869051 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 256/523] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:07 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/231350e78b2740a4b0aef761f7baa4e6/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:22:07.869946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:07.906597 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a6e75-2072-46d0-a072-1ad3015f1ed7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:07.964095 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:07.965091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:07.965607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:07.966063 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:07.966892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:07.968667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:07.992655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:07.993765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:07.994608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:07.995391 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 268/524] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 05:22:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:08.001906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:08.069122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68681a-e8b0-452d-a902-e45077cfa198 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:08.118240 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:08.118562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:08.118773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:08.119049 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] PATH_INFO: `/v3/services/network` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:08.119677 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:08.121099 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:08.121544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c022d-9017-4f1d-865c-70db010f0e50 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:08.133062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:08.134096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:08.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 257/525] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:07 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:08.144168 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:08.145300 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:08.169385 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:08.169929 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:08.170220 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:08.170732 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:08.171393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:08.171695 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:08.173252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:08.179503 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 269/526] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:22:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:08.185688 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:08.199550 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:08.276277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae054-3e55-494b-a53b-efb2d7467678 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:08.313762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745a5a1-c454-4618-b0b7-513c180b82fd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:08.316404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:08.317182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:08.317647 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:08.318086 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:08.318858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] RBAC: Adding query filter params (name=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:08.319324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:08.320838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:08.326211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 258/527] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:08.332260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:08.371135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:08.371776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:08.372052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:08.372319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:08.372939 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:08.373234 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:08.374596 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b6cde-81c4-4d6a-925d-991ba90018dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:08.387660 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 270/528] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:22:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:08.487818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388b2f-0cd5-4627-979e-9decca670e7b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:08.489399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:08.539126 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:08.539768 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:08.540048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:08.540315 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:08.540944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:08.541242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:08.542596 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0166a-0062-4cb8-81d3-4b4dce2a880e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:08.548872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 259/529] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:09.795518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f1519-67d4-41d4-8f99-72ccd2d74d91 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:09.796014 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f1519-67d4-41d4-8f99-72ccd2d74d91 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:09.796014 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f1519-67d4-41d4-8f99-72ccd2d74d91 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:09.797537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 271/530] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:09.803360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:09.803849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:09.804006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:09.832309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:09.833512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:09.911490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 260/531] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:22:10.199751 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f1519-67d4-41d4-8f99-72ccd2d74d91 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:10.259528 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:10.340022 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f1519-67d4-41d4-8f99-72ccd2d74d91 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:10.389286 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:10.389919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.390195 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.390462 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.420761 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:10.425654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:10.426630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:10.428412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 272/532] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:10.433364 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:10.553229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:10.563912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1359b4-16c0-47d2-82ec-022f6ae857c5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:10.608461 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:10.609074 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.609349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.609618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.610279 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:10.610575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:10.611950 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305f01-788d-47b8-9378-26cd32150a59 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:10.620356 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 261/533] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:10.628677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:10.631620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a853c93-b885-4e8b-aed1-cb5699c0e806 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.631620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a853c93-b885-4e8b-aed1-cb5699c0e806 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.633309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a853c93-b885-4e8b-aed1-cb5699c0e806 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.633496 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 262/534] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:10.641730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c7ab9-7d84-4951-9187-f0bb630a6d2f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.641730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c7ab9-7d84-4951-9187-f0bb630a6d2f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.641730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c7ab9-7d84-4951-9187-f0bb630a6d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.664181 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-169c7ab9-7d84-4951-9187-f0bb630a6d2f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:10.665644 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-169c7ab9-7d84-4951-9187-f0bb630a6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:10.743107 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 263/535] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:22:10.825288 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:10.829145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc134-32e1-4f7c-8c94-b7fad974ed22 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:10.866946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07763f7-cde0-4f1e-b9ef-5b9e9752e4ae None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.866946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07763f7-cde0-4f1e-b9ef-5b9e9752e4ae None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.866946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07763f7-cde0-4f1e-b9ef-5b9e9752e4ae None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.866946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 264/536] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:10.870363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:10.871010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.871309 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.871573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b444e-8cf2-4019-b1ad-ec0d155848ab None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:10.871686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.872101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b444e-8cf2-4019-b1ad-ec0d155848ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:10.872579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b444e-8cf2-4019-b1ad-ec0d155848ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:10.893370 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:10.893740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-aa7b444e-8cf2-4019-b1ad-ec0d155848ab None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:10.894749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:10.895448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-aa7b444e-8cf2-4019-b1ad-ec0d155848ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:10.920803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:10.921802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:10.922686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:10.922686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:10.922686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:10.925302 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 273/537] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:10.932035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.026835 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 265/538] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:11.079890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aba80-b5a8-4911-aa38-71f107e4ed20 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.080755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aba80-b5a8-4911-aa38-71f107e4ed20 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.081232 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aba80-b5a8-4911-aa38-71f107e4ed20 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.084270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 266/539] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:11.090299 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7f41-7437-4ca7-a1a3-0eb18bab0d6f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.090626 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.090910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.091182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.128319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:11.132198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:11.132996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.135831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.136381 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.136940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.137807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:11.139089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:11.144142 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:11.155725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.159053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 274/540] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:11.160149 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.183324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.219599 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 267/541] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:22:11.221006 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.305391 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373732c-a557-4683-982d-a7f6ef58d8cc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.324888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded145d-d767-43db-ae7f-7d92c8b94fb3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.347634 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00207892-ed76-4a77-bc4c-ce5006cc44cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.348384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00207892-ed76-4a77-bc4c-ce5006cc44cc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.349299 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00207892-ed76-4a77-bc4c-ce5006cc44cc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.349822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00207892-ed76-4a77-bc4c-ce5006cc44cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.360980 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.361131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00207892-ed76-4a77-bc4c-ce5006cc44cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:11.361603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.361877 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.362140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-00207892-ed76-4a77-bc4c-ce5006cc44cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:11.365678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:11.366594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:11.367844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 275/542] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:11.368572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde3d0b-ef2a-4cd3-974e-65e283023807 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.368572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde3d0b-ef2a-4cd3-974e-65e283023807 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.368756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde3d0b-ef2a-4cd3-974e-65e283023807 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.369940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:11.370055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 276/543] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:11.370632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde3d0b-ef2a-4cd3-974e-65e283023807 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.371954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7f1ba0b5-d7c5-4696-a200-ef015ec6f7d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:11.385317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:11.386283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:22:11.387219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:11.388471 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 268/544] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:11.389615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.389890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.390158 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.408296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:11.411746 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:11.413040 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.457571 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.481918 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 269/545] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:22:11.482630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.509505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde3d0b-ef2a-4cd3-974e-65e283023807 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.545938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.546565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.546843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.547112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.557301 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:11.562492 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:11.563476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:11.564300 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 277/546] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:11.564983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.601626 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3133f-1597-4dec-a93d-127cba7c483c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.637723 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.638280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.638558 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.639435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.639565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:11.639824 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:11.641162 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:11.646816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.650211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 270/547] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:11.655414 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.671043 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b629d-30b2-46e6-b31c-57a40b65e504 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.700172 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.705095 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.705691 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.705967 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.706235 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.706959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:11.707239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:11.709246 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:11.730878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 278/548] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:11.730878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.763563 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a12b9-fd82-4e52-8127-3cda0d4b5850 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.796331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.796923 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.797197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.797465 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.798100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:11.799544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:11.832099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705c2ee-8974-4eb0-9eb6-f0ac0008fca0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:11.865213 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:11.865744 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:11.866068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:11.866343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:11.866988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:11.867194 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 271/549] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:11 2022] POST /identity/v3/services => generated 240 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:11.867626 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:11.868094 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.868659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:11.880923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 279/550] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:11.882951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:11.937582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.947658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:11.997620 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35289b73-da83-4692-854e-288eaff9c5f5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.003014 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeff4b-17d4-40c6-98cf-91a7a4a55be1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.037377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.037939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.038212 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.038477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.043018 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.043685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.043966 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.044859 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.049115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.050577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.057342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.059191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:12.066200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:12.067157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:12.068114 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:12.068114 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:12.068114 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:12.068114 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 272/551] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:12.069296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:12.079461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:12.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:12.081668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:12.081668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:12.081668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:12.081668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 280/552] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:12.082068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.175282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637852d-96a4-444d-aefb-69d3beda55ee None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.185464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89602d-0478-4039-8b1f-daf931b8a9d1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.185670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:12.188514 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:12.209820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.210440 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.210722 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.210994 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.211678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.213048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.221990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.222615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.222890 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.223160 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.233510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.234941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:12.249267 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:12.250252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:12.251143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:12.251143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:12.251143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:12.251143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 281/553] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:12.251143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.278559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 273/554] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:12 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:12.288381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.368169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6394-71f2-4ead-997a-2c0ddc011f6c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.430422 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.431658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.432703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.433429 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.434265 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:12.434950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.437333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.442153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85ba6-3c08-4452-baa0-5c0f155b695c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.455364 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 282/555] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:12.456780 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.481490 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:12.488115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.494732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.494732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.494732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.494732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:12.494732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.494732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.505536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 274/556] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:12.506515 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.541390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:12.582486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52a596-afe5-4891-8440-f1b1ded4778d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.627652 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.628720 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.629499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.630201 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.630992 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:12.631462 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.632150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfb527-0dde-4109-8c4f-4e0f4c2b3d85 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.633162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.648258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 283/557] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:12.649283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.674681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.675551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.675955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.676366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.686502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.688565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:12.701696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:12.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:12.703664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:12.707785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 275/558] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:12.708736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.737401 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:12.777890 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635bb0a-6505-4a53-b9d6-81875ee69009 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.821759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:12.824977 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.825996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.826745 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.827471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.835517 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8dc3a-f42a-423d-b1b5-8083b21c94d9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:12.836845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.839782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:12.853325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:12.854384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:12.855259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 284/559] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:12.856427 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.876605 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:12.877872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:12.878322 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:12.878765 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:12.887450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:12.889318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:12.901494 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:12.902490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:12.903864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 276/560] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:12.905376 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:12.972132 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40e352-bbba-4429-8fe4-7169988edc77 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.007334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.013665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.014285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.014574 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.015105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.015855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:13.016207 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.018327 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.024201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309f8ff-8ed1-4806-a23a-2f6d1c2598e0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.047389 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 285/561] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:13.048542 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.061178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.062106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.062623 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.063100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.063913 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:13.064370 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.065882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.083389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 277/562] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:13.083389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.095240 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.192995 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc69c-1dcf-4d40-8e2d-6d1cde30de74 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.197363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a42480-c276-419e-84ab-9a453e6b3e2c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.233613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.234587 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.235079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.235618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.236986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.237657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.238256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.238602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.240379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:13.240759 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.242276 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.248338 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.249937 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:13.264944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:13.265925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:13.266864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:13.266864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:13.266864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:13.269021 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 278/564] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:13.269129 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 286/564] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:13.270059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.270257 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.339607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.350222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.389800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e03e15-7650-419b-a930-260bc3beaaf7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.408828 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482210d5-f751-4e7f-8c9d-32c690c0238f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.433355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.433965 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.434239 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.434505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.458736 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.460122 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.461019 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.461736 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/b04f725dd19b4a0f9669840fba801b3f/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.464903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=b04f725dd19b4a0f9669840fba801b3f, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.467205 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.504988 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.506450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.527239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 287/565] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:13 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/b04f725dd19b4a0f9669840fba801b3f/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:22:13.537126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.615999 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.698827 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 279/566] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:22:13.703583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a549d76-f364-45a9-81ce-559bd2de1184 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.705956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.725972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.770093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.770695 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.771026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.771322 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.771981 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:13.772283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.774089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.791404 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 288/567] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:13.796316 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.852410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e23fef-cbd3-436e-baa4-64ab1f6fd6d6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.889895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.890557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.890793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.891156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.901967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.903546 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.910248 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016d744-b11d-4ce1-80f4-c641207c2248 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:13.919504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:13.920573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:13.921484 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:13.921484 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:13.921484 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:13.921484 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 280/568] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:13.928230 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:13.951388 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:13.952405 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:13.952890 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:13.953364 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:13.953844 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:13.962628 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:13.964367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:13.977324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:13.978355 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:13.979251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:13.981702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 289/569] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:13.987573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.054086 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:14.137324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba620e8f-34ac-4898-8d85-31e5c481650e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.165804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8795b-e3d0-4a10-8a40-d3e032ef8fa0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.176164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.177314 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.177699 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.177974 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.178615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:14.178918 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.180664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bcfc5-02b9-4069-a025-0d07f680cf4c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.196762 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 281/570] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:14.198187 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8e3e-893b-4073-8c6c-859ed817fea9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.198482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8e3e-893b-4073-8c6c-859ed817fea9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8e3e-893b-4073-8c6c-859ed817fea9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.200229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 282/571] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:14.205156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8e3e-893b-4073-8c6c-859ed817fea9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.214196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.214813 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.215098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.215408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.216134 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:14.216877 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.218670 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62ce6-eb7f-4998-95fc-c986c5cc526c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.233956 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 290/572] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:14.236024 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.236310 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.236583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.260994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:14.262166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:14.300751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:14.338830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 291/573] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:14.339663 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8e3e-893b-4073-8c6c-859ed817fea9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.339806 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.370771 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:14.386970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.387872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.388327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.388750 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.410533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.413057 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:14.438645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:14.439691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:14.440606 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:14.443353 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 283/574] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:14.452254 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.493891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5416b-f2f0-415b-ae6a-a66ab09e12fa None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.531613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.532429 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.532803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.533172 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/a12b53c818724d518233346ee25ba8e9/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.560308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=a12b53c818724d518233346ee25ba8e9, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.561768 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.613792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8916a-4c7c-44e6-95a1-fba9e2fca1d0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.644385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:14.649637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.650313 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.650600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.650873 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.651646 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:14.651953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.653425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.668366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 284/575] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:14.669322 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.690615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 292/576] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:14 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/a12b53c818724d518233346ee25ba8e9/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:22:14.691516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.814993 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:14.839812 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ca93a-ca85-4d11-8358-8b081a00f71c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.862621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b1af6-d0d0-4849-b2c0-a589c2f18a90 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:14.886734 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.887675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.888163 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.888785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] PATH_INFO: `/v3/services/network` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.889519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.891046 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-04140b82-9cc8-4c4a-b62e-d11a0272ee0b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:14.901840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:14.902926 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:14.903849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 285/577] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:14 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:14.904949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcf7bf-ab2e-4e92-8ed2-3981c05d09fc None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.904949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcf7bf-ab2e-4e92-8ed2-3981c05d09fc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.905936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcf7bf-ab2e-4e92-8ed2-3981c05d09fc None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.909802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:14.909802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.910046 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 286/578] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:14.910123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.910123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.910123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=231350e78b2740a4b0aef761f7baa4e6) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:14.910123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:14.910123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7507ed3-f73c-40ab-bfe6-eacb0e1e8da0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:14.911512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcf7bf-ab2e-4e92-8ed2-3981c05d09fc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:14.916863 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 293/579] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:14 2022] GET /identity/v3/role_assignments?user.id=231350e78b2740a4b0aef761f7baa4e6&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:14.917921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:14.918229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:14.918460 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:14.938832 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:14.940094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:14.980340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:15.028783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 294/580] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:15.036994 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcf7bf-ab2e-4e92-8ed2-3981c05d09fc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.073799 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.074411 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.074688 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.074957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.075678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] RBAC: Adding query filter params (name=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:15.075976 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.077332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.082591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 287/581] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:15.088001 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:15.195987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:15.248658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f02dc-1818-40de-ab78-292f88a0948b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.289252 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.289864 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.290143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.290415 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.291037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] RBAC: Adding query filter params (type=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:15.291390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.292751 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.302565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 295/582] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:15.308071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:15.348886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:15.440705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa5564-faa9-4d68-867c-f484f76b96ff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.463482 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ad009-ab25-43c7-8f71-c03dd8ed4789 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.476252 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:15.491910 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:15.507362 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.508006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.508285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.509560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.509560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.510542 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.512645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.513269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.513543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.516843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.516843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] RBAC: Adding query filter params (interface=public, service_id=0af5e7c4a6ea428fadbf6a3098720c50, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:15.516843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.518831 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:15.530411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:15.531523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:15.532441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:15.534037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 296/583] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:15.539092 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:15.541341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 288/584] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/endpoints?service_id=0af5e7c4a6ea428fadbf6a3098720c50&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:15.546827 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:15.731913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7c095-9ddb-43be-8ff5-8ddf5aff371f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.732773 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992dc70-8e7e-467f-9d4c-f5a25fd192e0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.803002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.803670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.803947 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.804218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.804823 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.806169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.808918 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.809520 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.809797 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.810066 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.810689 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:15.811049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.813589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.823070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 289/585] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/services => generated 772 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:15.823416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 297/586] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:15.829009 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:15.899709 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:15.941711 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b4d7e-b434-49ea-8140-d08e79dc8eeb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:15.979362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:15.980469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:15.980749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:15.981019 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:15.981664 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:15.982019 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:15.983487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8de0-066c-407b-ae65-568cfd72461f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:15.989339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 290/587] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:15.995135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:16.056217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:16.098129 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b60d7-c84f-4fd9-9c08-b3bc7e69ef57 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:16.098489 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b60d7-c84f-4fd9-9c08-b3bc7e69ef57 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:16.098738 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b60d7-c84f-4fd9-9c08-b3bc7e69ef57 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:16.100793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 291/588] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:16.109399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:16.109741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:16.110387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:16.112999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a45529-14b1-45ee-b4ca-5a5f8d859822 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:16.135055 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:16.136380 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:16.165759 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:16.166779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:16.167061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:16.167359 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:16.168318 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] RBAC: Adding query filter params (interface=public, service_id=1ca40dc3ad6348478067cee8a2e260ca, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:16.168627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:16.170254 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:16.180102 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 298/589] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:15 2022] GET /identity/v3/endpoints?service_id=1ca40dc3ad6348478067cee8a2e260ca&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:16.185499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:16.220569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 292/590] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:16.289755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb4ab0-6be7-4c94-a92b-bda204b3ae70 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:16.312086 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:16.324873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:16.325479 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:16.325754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:16.326023 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:16.326615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:16.328001 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:16.333303 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 299/591] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:16 2022] GET /identity/v3/services => generated 772 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:16.508352 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:16.556544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:16.619370 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7904bf-4d4f-4fd1-bf51-c30bbaeaf4d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:16.657741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:16.658747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:16.659227 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:16.659856 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:16.671320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:16.677804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:16.678737 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:16.679675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:16.682277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 293/592] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:16.689069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:16.823354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1aca1-e121-41eb-8085-029f3a01248c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:16.866354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:16.866967 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:16.867246 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:16.867539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:16.868200 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:16.868501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:16.869859 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:16.882476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 300/593] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:16.888962 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:17.038193 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe10c1-b3e5-4ae6-90fb-bdd53f9d9040 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:17.048755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:17.089237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:17.090545 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.091254 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.091561 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.104656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:17.106150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9a05695a-f822-4e71-b33d-9be10a871ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:17.132152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:17.133173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:17.134491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:17.134491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:17.134491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:17.134491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 294/594] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:17.140393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:17.205902 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:17.228670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32539986-0279-4f45-ba0f-fa6e4ca39300 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.228995 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32539986-0279-4f45-ba0f-fa6e4ca39300 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.229230 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32539986-0279-4f45-ba0f-fa6e4ca39300 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.230817 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 295/595] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:17.244782 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.245065 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.245360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.277450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610dab-65a6-4a9c-be94-2b5806b5e943 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:17.287819 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:17.289130 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:17.318728 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:17.319923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.321154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.321801 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.323722 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:17.324264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:17.326012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:17.348247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 301/596] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:17.355185 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:17.438242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 296/597] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:17.509128 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8071ba7-4eea-4212-bf5e-4b26fbd056a9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:17.511495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:17.562049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:17.562695 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.562974 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.563249 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.574716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:17.580253 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9259226d-e043-4ff1-8dbf-d7b786e7d8fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:17.593101 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:17.594061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:17.594982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 302/598] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:17.600686 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:17.657889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5291cb-b68b-49cf-9d77-8025fcbe1d85 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.658224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5291cb-b68b-49cf-9d77-8025fcbe1d85 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.658426 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5291cb-b68b-49cf-9d77-8025fcbe1d85 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.660112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 303/599] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:17.666897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.667210 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.667487 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.693641 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:17.694828 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:17.764202 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:17.792502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c625c22-1d10-4014-aa68-12cf2cdde76f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:17.823393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 304/600] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:17.824333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:17.845939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:17.846548 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:17.846824 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:17.847100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:17.847844 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:17.848149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:17.851103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:17.880591 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 297/601] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:22:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:17.887116 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:17.999195 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.014147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90fd8-ca18-4292-9bf5-a42135f57b02 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.046981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4717b6-9cb9-4306-b417-54a0d585cc39 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.080354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.080993 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.081271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.081544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.093502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:18.099441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:18.100363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:18.101260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 305/602] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:18.107790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.111979 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.112602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.112884 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.113154 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.149084 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.217341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.218889 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.232548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a763a0-4f8c-4c61-a476-86bfdacd8b09 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.269285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.269995 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.270444 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.270717 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.271483 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:18.271847 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.273277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.286696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 306/603] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:18.286696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.310537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.345634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 298/604] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:22:18.345634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.392760 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.427698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2b5c-8178-4ac0-a6e2-4cc042415115 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.462536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66029c-e2ce-42e6-a4dd-5d90ad275a0c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.463500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.464624 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.465073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.465566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.466363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.468028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:18.478436 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:18.479418 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:18.480326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:18.487702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 307/605] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:18.488918 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.499106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.500045 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.500179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.500406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.512203 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.513696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:18.529469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:18.530463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:18.531442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:18.531442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:18.531442 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:18.535707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 299/606] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:18.537744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.585342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.606360 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899807ac-409c-4864-b17e-c9ca242dd6d8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.641025 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.648446 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.648676 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef112a-0c49-4f98-9b91-26966953dc34 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.649525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.650112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.650566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.669602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.671866 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.686391 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.687003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.687280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.687580 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.688205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:18.688497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:18.689499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:18.691060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:18.691060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:18.691060 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:18.691234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.691340 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 308/607] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:18.691340 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.695206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda26037-258e-40ce-91c8-d2b4556a7a4e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.705259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 300/608] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:18.706740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8183-ea8f-4a8d-a55f-3d221e84c274 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.707025 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8183-ea8f-4a8d-a55f-3d221e84c274 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.707327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8183-ea8f-4a8d-a55f-3d221e84c274 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.709180 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 301/609] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:18.709726 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b8183-ea8f-4a8d-a55f-3d221e84c274 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.805820 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65bcb9-660b-498f-af77-fdd11337db1f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.819514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b8183-ea8f-4a8d-a55f-3d221e84c274 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.836158 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.841400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.842034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.842337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.842615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.843315 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:18.843638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.845058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.862603 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 309/610] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:18.862603 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:18.864615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:18.865266 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.865618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.865938 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.866633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:18.866946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:18.868731 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e9d18-0b1f-4712-8721-5934e299e1d3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:18.881796 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 302/611] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:18.883035 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:18.883415 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:18.883701 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:18.905242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:18.906897 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:18.925263 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:18.975508 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f1c74-1c1e-490f-b8da-966ecc081234 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:18.983130 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 303/612] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:18.984062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.009665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.010557 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.010831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.011484 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:19.011789 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.013143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.031339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 310/613] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:19.031339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.081749 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.112072 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcab6-778e-47f9-8bae-e80f131c3334 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.140465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840dc08-4717-4a16-a5e4-23c6d80531a2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.154842 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.155693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.155955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.156155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.165749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.167568 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c79ed7b6-852e-43fd-b3ec-973a128786cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:19.182544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:19.183612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:19.184553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 304/614] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:22:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:19.185684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71eac-cea4-4db0-bd08-f5fa69b8b609 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.185684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71eac-cea4-4db0-bd08-f5fa69b8b609 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.185841 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71eac-cea4-4db0-bd08-f5fa69b8b609 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.191476 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 305/615] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:19.193582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71eac-cea4-4db0-bd08-f5fa69b8b609 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.204073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.204728 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.205009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.205282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.211247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.212729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.214316 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3168db79-f606-4cb9-b838-b336c1780274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:19.227640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:19.228667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:19.229575 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 311/616] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:19.230079 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.230891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.230891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.254270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:19.255583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:19.320147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71eac-cea4-4db0-bd08-f5fa69b8b609 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.335327 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 312/617] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:19.335327 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.367891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.368498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.368774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.369044 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.369761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:19.370066 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.371631 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.386117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.391368 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 306/618] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:19.392439 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.446920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36038b4e-356d-40a8-8c84-a8ce0b5304d1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.480864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.485475 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.486281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.486563 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.486873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.487602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:19.491338 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.491338 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.507384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 313/619] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:19.507384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.526380 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef2fb4-4c26-4e29-9e1a-7c27d7b50a23 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.567668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.568304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.568588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.568864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] PATH_INFO: `/v3/services/network` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.569450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.570812 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:19.579830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:19.580880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:19.581720 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 307/620] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:19.582501 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.611349 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfaa63-9dd1-417e-ada5-3f435f9d91d2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.618489 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.646984 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.647853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.648280 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.649585 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.689320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885082f-ceb3-405a-a4a3-0d168294b79c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.719059 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.721010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.722544 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.725120 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a853-314b-4913-9060-cc303f245068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.725731 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a853-314b-4913-9060-cc303f245068 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.726015 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a853-314b-4913-9060-cc303f245068 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.726287 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793a853-314b-4913-9060-cc303f245068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.727094 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793a853-314b-4913-9060-cc303f245068 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=b04f725dd19b4a0f9669840fba801b3f) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:19.727460 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793a853-314b-4913-9060-cc303f245068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.731511 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793a853-314b-4913-9060-cc303f245068 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.735062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 308/621] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/role_assignments?user.id=b04f725dd19b4a0f9669840fba801b3f&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:19.735816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a853-314b-4913-9060-cc303f245068 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.848810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 314/622] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:22:19.849583 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.858954 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:19.888559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793a853-314b-4913-9060-cc303f245068 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:19.935763 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:19.936764 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:19.937047 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:19.937324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:19.937959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] RBAC: Adding query filter params (name=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:19.938723 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:19.940585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:19.947008 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 309/623] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:19.950596 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:19.973960 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fddaf7-dac1-4a10-b7d5-63c0ea9e3338 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.011441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.012194 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.012476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.012752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.013328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.014807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.015167 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:20.025010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:20.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:20.037619 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 315/624] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:20.038427 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.066792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c529b9e-659c-4628-a90d-3cc9747bb41f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.103996 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.104965 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.105381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.105817 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.108259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.120736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.122762 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:20.140878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:20.142003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:20.142003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:20.142003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:20.142446 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065b17b-0a8d-40e1-b416-8834966d40c1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.143816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 310/625] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:20.146118 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.181466 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.182029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.182406 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.182776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.183553 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] RBAC: Adding query filter params (type=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:20.184009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.185520 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.195671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 316/626] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:20.196622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.265673 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f50d42-b1a3-400e-809b-0fd2b99bbb35 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.281697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.307860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc627a-aac7-4bf5-957e-95e63ea61024 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.311020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.311875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.312298 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.312713 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.313500 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:20.313962 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.315515 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.321387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 311/627] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:20.322378 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.360752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.361523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.361941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.362313 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.363076 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:20.363504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.365529 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.367669 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.384372 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 317/628] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:20.386024 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.446400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841e639-c83b-4615-87c2-1160fa4b716e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.485325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.486831 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.487341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.487817 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.489746 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.492178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.512635 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d3898-37fc-4245-8222-51b2fc551c9b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.548072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.548678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.548958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.549229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.549846 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:20.550152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.552074 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.561719 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 318/629] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:20.561719 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.602152 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 312/630] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:20 2022] POST /identity/v3/endpoints => generated 337 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:20.603945 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.624936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.649977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.680581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a17d4-7c52-4098-90e1-0b9e919bd4b2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.729589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.730958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.731247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.731550 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.741909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.743396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:20.755651 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:20.756636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:20.757543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 319/631] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:20.761426 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345004a-d63f-41e1-800b-69c60b233cfb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.763224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:20.813614 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.815164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.815949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.816401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.817583 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.819745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.882633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3692-9bcc-4d24-b529-61a2e8f1d28f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:20.898066 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:20.903287 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 313/632] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:20.930694 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:20.931357 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:20.931633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:20.931907 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:20.932658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:20.932964 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:20.934501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845466c-f755-463e-b4dc-964d89d61d6c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:20.949262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 320/633] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:20.955490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:21.069531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:21.089983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07a70-5933-4320-9bbc-707e6650bb95 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:21.128445 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:21.129122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:21.129365 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:21.130509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:21.131343 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=a12b53c818724d518233346ee25ba8e9) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:21.131898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:21.133795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87336a-b8ca-4bcd-8b2b-2ee1db874af9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:21.140554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 314/634] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:20 2022] GET /identity/v3/role_assignments?user.id=a12b53c818724d518233346ee25ba8e9&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:21.471613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44dfe8-a874-4c4e-b44c-6c4585cdedfa None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:21.471613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44dfe8-a874-4c4e-b44c-6c4585cdedfa None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:21.471613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44dfe8-a874-4c4e-b44c-6c4585cdedfa None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:21.472814 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 321/635] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:21.479051 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26c444-975f-4431-afa0-12b094ac2eeb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:21.479623 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26c444-975f-4431-afa0-12b094ac2eeb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:21.480054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26c444-975f-4431-afa0-12b094ac2eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:21.499141 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ab26c444-975f-4431-afa0-12b094ac2eeb None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:21.500604 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ab26c444-975f-4431-afa0-12b094ac2eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:21.570012 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 315/636] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:21.837967 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44dfe8-a874-4c4e-b44c-6c4585cdedfa None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:21.948006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44dfe8-a874-4c4e-b44c-6c4585cdedfa None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:21.988463 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fef44-b459-47d0-8943-6733f60b0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:21.989891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fef44-b459-47d0-8943-6733f60b0cd6 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:21.990325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fef44-b459-47d0-8943-6733f60b0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:21.990755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fef44-b459-47d0-8943-6733f60b0cd6 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:21.991589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fef44-b459-47d0-8943-6733f60b0cd6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:21.993157 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fef44-b459-47d0-8943-6733f60b0cd6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:22.045599 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 322/637] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:21 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:22.074685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771deb-2e01-40be-ad9d-9c15f9e3b5b9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:22.074967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771deb-2e01-40be-ad9d-9c15f9e3b5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:22.075234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18771deb-2e01-40be-ad9d-9c15f9e3b5b9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:22.077102 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 316/638] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:22.083179 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:22.083536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:22.083835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:22.107182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:22.108645 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:22.137521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:22.194486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 323/639] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:22 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:22.475721 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18771deb-2e01-40be-ad9d-9c15f9e3b5b9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:22.570429 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:22.594210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18771deb-2e01-40be-ad9d-9c15f9e3b5b9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:22.629590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:22.630265 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:22.630547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:22.630818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:22.631415 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:22.632761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:22.641523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:22.642594 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:22.643527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 317/640] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:22.651244 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:22.758490 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88250a-e1b2-4673-b3d2-037eb87a617f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:22.793863 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:22.794478 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:22.794755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:22.795027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:22.795689 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:22.795988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:22.797490 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa4116-ddc2-4344-8f78-83d0a4a44497 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:22.840892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 324/641] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:22.846557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:22.952730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7519bc-c95a-4e00-8c74-27c96c7acd64 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:22.987399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:22.988323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:22.988898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:22.989362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:22.990171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:22.990642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:22.992179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a706c-7e8a-444c-b48c-96fb1a267910 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:22.997336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 318/642] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:23.217108 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4edcb-111f-4266-9bd2-33474c653331 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.217405 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4edcb-111f-4266-9bd2-33474c653331 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.217671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4edcb-111f-4266-9bd2-33474c653331 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.219361 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 325/643] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:23.224783 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725afea-1147-4303-87ec-86b6308750f5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.225070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725afea-1147-4303-87ec-86b6308750f5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.225347 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725afea-1147-4303-87ec-86b6308750f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.244633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4725afea-1147-4303-87ec-86b6308750f5 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:23.245814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4725afea-1147-4303-87ec-86b6308750f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:23.248105 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:23.321590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 319/644] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:23 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:23.405797 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd3c6-0e16-406c-9d0d-75b18b0d8ac3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.406090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd3c6-0e16-406c-9d0d-75b18b0d8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.406411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bd3c6-0e16-406c-9d0d-75b18b0d8ac3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.408052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 326/645] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:23.417502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.417795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.418077 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.436864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:23.438057 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:23.506558 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 320/646] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:23 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:23.599325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bd3c6-0e16-406c-9d0d-75b18b0d8ac3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:23.621713 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:23.706941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bd3c6-0e16-406c-9d0d-75b18b0d8ac3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:23.742835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:23.746459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.746459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.746459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.746459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:23.746459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:23.756337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:23.757600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:23.758682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:23.761081 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 327/647] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:23.766890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:23.795746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:23.841431 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:23.878569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe2a5-6a5c-4a05-9f45-ddf0dffe48e3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:23.887634 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:23.903734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afe339-6b04-4933-bc5b-995f2ddeb4f8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:23.916212 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:23.916838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.917120 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.917400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.918034 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:23.918341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:23.919730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:23.929668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 321/648] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:23.930927 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:23.940945 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:23.941594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:23.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:23.942224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:23.955094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:23.960985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:23.961938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:23.962795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:23.967148 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 328/649] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:22:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:23.968134 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.037413 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330d6e0-6c8d-4531-b84f-830ebed444a9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.071026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:24.079914 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.085430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.085430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.085430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.085430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:24.085430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.085430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.087941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5dac3-4a35-4400-99f9-0f20c9feeff1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.094477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 322/650] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:24.129182 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.130309 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.130796 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.131284 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.132196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:24.132707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.134319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.144477 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 329/651] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:22:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:24.151138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.263769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d2660-82e6-4d1e-8e05-0bdfd1396b9f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.305094 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.305986 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.306272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.306542 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.317552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.318961 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c28bb8be-9b62-4fde-be45-563a4df4775b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:24.334560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:24.335651 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:24.337278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:24.337278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:24.337278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:24.337278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 323/652] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:24.341671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.352745 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:24.371359 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba62b14-9b06-47c2-bb00-5cd3e006a96c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.373432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba62b14-9b06-47c2-bb00-5cd3e006a96c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.373432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba62b14-9b06-47c2-bb00-5cd3e006a96c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.373999 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 324/653] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:24.379198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.380210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.380637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.399204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:24.400633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:24.448365 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:24.462978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22946eda-4f20-4c1c-8492-c31b2e923e78 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.479018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 325/654] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:24 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:24.499134 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.499765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.500042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.500311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.500938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:24.501236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.502588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.514752 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 330/655] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:24.520313 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.623017 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579016f9-5da3-47f2-91f0-935e83b56aad None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.664159 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.664794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.665075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.665346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.672625 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.674052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:24.685581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:24.686565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:24.687586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 326/656] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:24.694654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.764928 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:24.765543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.800923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053163-c532-47c3-8223-2cfd39d88f19 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.809482 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:24.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.838838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.839114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.839413 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.840135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:24.840435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.841919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.861853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 331/657] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:24.870205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c57e55-8e6e-46de-82be-34fc29683559 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:24.876443 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:24.906853 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:24.907520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:24.907800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:24.908073 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:24.908656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:24.910195 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:24.920096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:24.921081 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:24.922026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 327/658] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:24.930662 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.008049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162eb5-f07d-4492-ace5-47ba0fb8cf87 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.032980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.044741 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.045830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.046324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.046787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.057815 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.058702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a867d36-ef2d-472f-a76c-5258dc1a2a90 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.059831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.060367 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:25.079382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:25.080382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:25.081280 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:25.081280 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:25.081280 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:25.083579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 332/659] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:25.092533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.105687 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.106298 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.106573 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.106844 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.107510 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:25.107807 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.109171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.114640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 328/660] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:25.120155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.231927 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c056f0-75e8-4315-b5df-008922971dd0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.233566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827cfbe-6912-47a1-8a99-ce770eae0497 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.267062 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.267671 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.267948 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.268214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.268891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:25.269192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.270885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.280927 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 329/661] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:25.281738 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.281868 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.282782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.283277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.283822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.284998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:25.287251 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.288951 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb2cd0-c876-4bfd-820e-e7fe16558e78 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.303339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 333/662] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:25.304844 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5da54-6a62-4498-881c-41e001d15da2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.305130 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5da54-6a62-4498-881c-41e001d15da2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.305404 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5da54-6a62-4498-881c-41e001d15da2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.306691 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 334/663] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:25.311248 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5da54-6a62-4498-881c-41e001d15da2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.319717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.349127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.398528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3291a7-db7c-48ad-bf4c-5da1a1945fdb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.416811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5da54-6a62-4498-881c-41e001d15da2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.441271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.441830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.442112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.442390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.443268 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] RBAC: Adding query filter params (interface=public, service_id=5cb4be88597d4a2fa563aaf215d1d0da, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:25.443928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.445590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffee3f-5a09-4e46-8cd0-e681e1a79c10 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.455091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.455256 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 330/664] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:25 2022] GET /identity/v3/endpoints?service_id=5cb4be88597d4a2fa563aaf215d1d0da&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:25.455390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.455390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.455390 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.455608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.455608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.455608 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.463090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.464573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.472470 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:25.473724 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:25.475808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:25.476709 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:25.477604 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:25.481634 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 335/665] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:25.482533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.543995 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 331/666] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:25 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:25.544881 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.581036 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.600093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.603641 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47490-d521-4910-aa99-86ce442ccad9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.647803 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.648990 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.649850 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.650335 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.651152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.653415 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.661281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 336/667] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:25 2022] GET /identity/v3/services => generated 1029 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:22:25.668952 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37187b-9c9e-4717-9d14-31a17522d7ec None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.709748 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.710778 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.711274 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.711781 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.712722 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:25.713246 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.714920 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.729900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 332/668] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:25.739408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.829969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:25.861057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e96cfa-9a3c-460a-a1c5-b701d9565397 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.909193 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225547c2-1c95-4749-ab2d-bd4239a06d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.911247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225547c2-1c95-4749-ab2d-bd4239a06d4f None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.911977 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225547c2-1c95-4749-ab2d-bd4239a06d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.912445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225547c2-1c95-4749-ab2d-bd4239a06d4f None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/a12b53c818724d518233346ee25ba8e9/roles/40bd8db9aded43d38ac338975828fc90` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.915606 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225547c2-1c95-4749-ab2d-bd4239a06d4f None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=a12b53c818724d518233346ee25ba8e9, role_id=40bd8db9aded43d38ac338975828fc90)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.917411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225547c2-1c95-4749-ab2d-bd4239a06d4f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:25.936853 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:25.952575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6597c66-d728-4441-b606-3b9d087cf807 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:25.991057 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be0f0e-cd77-4e62-9d5a-7ec6df377326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:25.991637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be0f0e-cd77-4e62-9d5a-7ec6df377326 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:25.991899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be0f0e-cd77-4e62-9d5a-7ec6df377326 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:25.992168 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be0f0e-cd77-4e62-9d5a-7ec6df377326 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:25.992808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be0f0e-cd77-4e62-9d5a-7ec6df377326 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:25.995223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 337/669] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:25 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/a12b53c818724d518233346ee25ba8e9/roles/40bd8db9aded43d38ac338975828fc90 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:22:25.999913 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be0f0e-cd77-4e62-9d5a-7ec6df377326 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:26.103483 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 333/670] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:25 2022] POST /identity/v3/services => generated 241 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:26.366861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed2e1-32de-4908-9859-b2e0d5ab2685 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:26.367143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed2e1-32de-4908-9859-b2e0d5ab2685 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:26.367447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed2e1-32de-4908-9859-b2e0d5ab2685 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:26.369029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 338/671] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:26.374552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e727c-ac24-4912-8f3c-e0128ca89c6d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:26.375035 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e727c-ac24-4912-8f3c-e0128ca89c6d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:26.375504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e727c-ac24-4912-8f3c-e0128ca89c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:26.394427 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8a8e727c-ac24-4912-8f3c-e0128ca89c6d None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:26.396419 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8a8e727c-ac24-4912-8f3c-e0128ca89c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:26.475768 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 334/672] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:26 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:26.750443 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ed2e1-32de-4908-9859-b2e0d5ab2685 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:26.768581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:26.852270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ed2e1-32de-4908-9859-b2e0d5ab2685 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:26.889123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffed84-894d-4b89-a777-cab1e626e795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:26.889761 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffed84-894d-4b89-a777-cab1e626e795 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:26.889987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffed84-894d-4b89-a777-cab1e626e795 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:26.890255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffed84-894d-4b89-a777-cab1e626e795 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:26.890901 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffed84-894d-4b89-a777-cab1e626e795 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:26.892298 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffed84-894d-4b89-a777-cab1e626e795 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:26.977224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 339/673] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:26 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:27.906297 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a77a1-944d-490e-be75-a5bd06adf153 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:27.906866 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a77a1-944d-490e-be75-a5bd06adf153 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:27.907148 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a77a1-944d-490e-be75-a5bd06adf153 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:27.908896 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 335/674] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:27.915194 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1983e8-d33b-4b63-982e-09fb08aba601 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:27.915656 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1983e8-d33b-4b63-982e-09fb08aba601 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:27.915982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1983e8-d33b-4b63-982e-09fb08aba601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:27.934408 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1b1983e8-d33b-4b63-982e-09fb08aba601 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:27.935608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1b1983e8-d33b-4b63-982e-09fb08aba601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:28.009804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 340/675] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:28.205472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f083aa9-c283-408a-828e-234a43c98516 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.206102 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f083aa9-c283-408a-828e-234a43c98516 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.206589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f083aa9-c283-408a-828e-234a43c98516 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.209189 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 336/676] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:28.215277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48021-bcd9-4bc5-adf8-bfa68de98b54 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.215937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48021-bcd9-4bc5-adf8-bfa68de98b54 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.216443 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48021-bcd9-4bc5-adf8-bfa68de98b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.230341 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:28.237657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-46e48021-bcd9-4bc5-adf8-bfa68de98b54 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:28.239258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-46e48021-bcd9-4bc5-adf8-bfa68de98b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:28.284241 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f083aa9-c283-408a-828e-234a43c98516 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.327325 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 341/677] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:28.340162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e85b40-9ded-4a5e-95b5-37c96315dd50 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.340462 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e85b40-9ded-4a5e-95b5-37c96315dd50 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.340715 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e85b40-9ded-4a5e-95b5-37c96315dd50 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.342316 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 342/678] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:28.347560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce414a23-32af-428c-97de-f9e00170467e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.347846 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce414a23-32af-428c-97de-f9e00170467e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.348154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce414a23-32af-428c-97de-f9e00170467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.366645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:28.368429 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ce414a23-32af-428c-97de-f9e00170467e None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:28.369837 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ce414a23-32af-428c-97de-f9e00170467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:28.401932 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f083aa9-c283-408a-828e-234a43c98516 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:28.452960 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 343/679] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:28.462681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:28.463257 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.463564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.463833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.464474 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:28.465832 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:28.474936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:28.475882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:28.476759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 337/680] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:22:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:28.482099 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce414a23-32af-428c-97de-f9e00170467e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.548623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:28.586028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce414a23-32af-428c-97de-f9e00170467e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:28.620292 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:28.621128 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.621553 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.622219 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.622986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:28.623125 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.623584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:28.625090 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:28.630502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 344/681] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:28.635679 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.736319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8590ecd-5a86-427a-9556-107f299644ff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:28.736994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dcecb-a7e0-4164-a283-06bc52035327 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:28.768935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:28.775127 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:28.776060 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.776516 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.776942 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.777644 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:28.778231 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.778514 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.778790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.779451 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:28.779754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:28.780329 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:28.781111 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:28.786187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 345/682] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:28.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.795838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:28.800693 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:28.801907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:28.802785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 338/683] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:22:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:28.805756 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.893940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada0780-6e17-4fea-9960-6e80cd58a3e2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:28.912148 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d6c65-5371-489e-90cb-ddf7820e3f69 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:28.927420 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:28.928043 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.928323 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.928597 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.929171 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:28.930530 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:28.939104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:28.940123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:28.940978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 346/684] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:28.942306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:28.953317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:28.954223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:28.954736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:28.955182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:28.956061 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:28.958208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.000947 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.043340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 339/685] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:29.043340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.046032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24759ce-c56b-402d-8b97-aca97e3f5a0c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.093854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.094472 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.094756 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.095023 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.095767 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:29.096073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.097442 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.105529 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.106459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 347/686] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:22:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:29.116437 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.178784 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d9a7f-2f15-49af-a227-50cfab78f1cd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.216318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.216937 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.217219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.217493 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.218124 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:29.218425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.219824 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b252e-f861-49d6-96d2-39dd4e7275c6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.224939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 340/687] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:29.227330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13da12-a1d9-4ada-b6b2-a13ac4bb21d3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.227330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13da12-a1d9-4ada-b6b2-a13ac4bb21d3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.227330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13da12-a1d9-4ada-b6b2-a13ac4bb21d3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.228037 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 341/688] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:29.231222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13da12-a1d9-4ada-b6b2-a13ac4bb21d3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.240912 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36447b7-5aa4-4756-9bf8-622238b0e7c6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.260856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.279652 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.280286 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.280566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.280839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.292843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.294743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6ce8a5fa-0148-40c2-9f6f-3f2a288a8c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:29.309838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:29.310962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:29.311913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:29.311913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:29.311913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:29.311913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 348/689] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:29.312226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.312510 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.312783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.331126 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:29.332837 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:29.353003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13da12-a1d9-4ada-b6b2-a13ac4bb21d3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.368893 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.390890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.391634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.391899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.392173 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.392787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:29.393082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.394679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.404565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 349/690] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:29.404565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.407614 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 342/691] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:29.409951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.510017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.514223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769e432-9965-4815-84b2-123e6a6e68b5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.530115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0681e6-12cd-41d2-8e74-6b066e8bfc21 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.553324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.553935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.554213 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.554547 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.555718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:29.556215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.557887 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.571462 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 350/692] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:29.571828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.572437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.572730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.573005 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.573701 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] RBAC: Adding query filter params (interface=public, service_id=c98b0a24c18b4b4c804c82dbc7f95eb4, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:29.574010 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.580654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.580870 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.580957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 343/693] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/endpoints?service_id=c98b0a24c18b4b4c804c82dbc7f95eb4&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:29.588206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.640930 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.705140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f49597-1a28-41ea-8984-7e4c97694f25 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.712339 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81171d83-f053-4a8b-8bb0-7b066e2950dd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.745627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.748220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.748220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.748220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.748486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.748577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.748648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.748648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.748648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.749893 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.756526 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 344/694] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:22:29.756815 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.757744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.758333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:29.778937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:29.779980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:22:29.780958 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:29.785008 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 351/695] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:29.786504 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.807469 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.899599 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34531f3-bd23-4685-8f8d-aaee548f3829 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.910445 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a5125-8c62-4254-9702-4a961b967867 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:29.921542 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:29.938677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.939396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.939618 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.939894 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.940632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:29.940935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.943007 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.965660 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 352/696] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:29.968113 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:29.969050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:29.969524 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:29.969998 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:29.970788 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:29.971171 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:29.972575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:29.985289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:29.986337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:29.987430 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 345/697] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:29.993672 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.070358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:30.093075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af80ad2-43e6-4ef6-8e07-1311e7bde123 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.124087 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e495cd-92dd-4a3f-8f14-694954376841 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.129729 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.130573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.131050 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.131560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.166723 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.167631 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.167898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.168781 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:30.169086 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.171363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.181994 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 346/698] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:30.183448 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.210057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.212112 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.227424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:30.305697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:30.309515 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a4d1b-8101-4b6d-a87e-e3568eb45190 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.334775 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 353/699] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:22:30.339960 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.359790 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.361018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.361853 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.362358 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.363195 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:30.364284 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.366742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.373973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 347/700] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:30.379278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.455447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a6e3-8862-4460-82f8-36933bfb0f3a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.479382 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:30.485465 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b4e4e-172f-4f41-b73c-ad509c9b5f2f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.498778 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.499506 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.499789 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.500092 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.513372 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.515139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.530226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.531148 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.531678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.532160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.533050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] RBAC: Adding query filter params (interface=public, service_id=d6cf097d0dcb42dc987e7fdefb05bbff, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:30.533283 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:30.534206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:30.535214 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:30.535214 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:30.535214 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:30.536626 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.538245 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.539368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 354/701] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:30.540214 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.546198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 348/702] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/endpoints?service_id=d6cf097d0dcb42dc987e7fdefb05bbff&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:30.551336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.582020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:30.658775 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d84f13-a139-4aa0-ba8a-1ea17a99527c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.685538 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f6b2-d290-4f02-a434-f2602bf0294b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.699818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.700787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.701209 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.701690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.702441 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.704302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.711912 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 349/703] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 05:22:30.730750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.731601 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.732034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.732405 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.733396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:30.733806 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.735423 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.749943 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 355/704] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:30.756539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:30.790757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:30.873007 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172a2c1-f9e6-46e9-a149-6f9fb342dcfc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:30.909704 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:30.910285 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:30.910571 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:30.910846 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:30.918089 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:30.919564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:30.931559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:30.932903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:30.933774 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 350/705] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:30.940497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:31.034089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:31.046826 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112941a1-a290-453b-8a66-5e867c1c48db None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:31.085696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:31.086307 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:31.086585 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:31.086856 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:31.087599 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:31.087910 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:31.089333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa0bc9-b30f-4f02-b10c-8515114965a4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:31.101885 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 356/706] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:31.107874 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:31.202795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:31.219557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29728f-de22-46cf-b13a-0c753db25a34 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:31.255983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:31.256919 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:31.257424 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:31.257890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:31.258793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] RBAC: Adding query filter params (role.id=40bd8db9aded43d38ac338975828fc90, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=a12b53c818724d518233346ee25ba8e9) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:31.259322 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:31.261054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319390ed-af3e-4054-b432-92273cd451a5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:31.268892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 351/707] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:31 2022] GET /identity/v3/role_assignments?user.id=a12b53c818724d518233346ee25ba8e9&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=40bd8db9aded43d38ac338975828fc90 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:31.297946 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a11f2c-d528-440c-8481-1d94be49c09f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:31.298229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a11f2c-d528-440c-8481-1d94be49c09f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:31.298509 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a11f2c-d528-440c-8481-1d94be49c09f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:31.300284 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 357/708] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:31.306792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:31.307088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:31.307377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:31.330477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:31.331856 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:31.423431 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 352/709] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:31.691291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a11f2c-d528-440c-8481-1d94be49c09f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:31.799046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a11f2c-d528-440c-8481-1d94be49c09f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:31.836188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:31.837171 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:31.837674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:31.838135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:31.838896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:31.840516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c9211fe3-00a4-4c17-a8ba-79ce32e28b6e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:31.853700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:31.854707 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:31.855570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 358/710] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:31.862970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:31.950473 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:31.971433 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f0df2-f10d-48da-8d71-3c9fef03a2bf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:31.992871 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74981bf3-35b1-4cee-8f76-62ec2c37ce55 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:31.993421 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74981bf3-35b1-4cee-8f76-62ec2c37ce55 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:31.993421 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74981bf3-35b1-4cee-8f76-62ec2c37ce55 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:31.994790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 359/711] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:31.999847 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:32.000128 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:32.000405 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:32.013183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:32.013894 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:32.014170 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:32.014454 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:32.015063 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:32.015391 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:32.016740 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:32.019239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:32.021164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:32.022500 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 353/712] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:32.028106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:32.029337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:32.107393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 360/713] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:32.144076 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291d9e-d71d-4ea4-9f54-a35b5b70a158 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:32.180640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:32.181308 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:32.181588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:32.181913 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:32.182561 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:32.182863 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:32.184280 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:32.190242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 354/714] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:32.388924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:32.497430 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295fe97-3135-4a2a-abdb-44dbaeb26c64 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:32.536915 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:32.537558 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:32.537837 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:32.538108 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:32.538690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:32.540109 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:32.549314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:32.550317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:32.551170 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 361/715] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:32.556640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:32.568589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:32.660359 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be66a0b-7e93-42b0-86fc-78246f27b2b1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:32.695493 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:32.696156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:32.696440 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:32.696712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:32.697338 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:32.697636 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:32.698971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:32.703977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 355/716] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:32.710180 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:32.720162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:32.815845 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566daca-1776-49be-a615-ad4f1d124c20 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:32.851445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:32.852052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:32.852334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:32.852606 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:32.853226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:32.853525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:32.854952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f6216-d224-4d70-b123-2c36f78ae295 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:32.860254 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 362/717] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:32.866500 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:32.943233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:32.973218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d42bc-4d31-4839-afcd-fb8449b8f551 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:33.008736 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:33.009349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.009629 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.009901 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.010577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:33.011966 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:33.042974 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0008351-09f7-42d0-9290-5591f5b64dba None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.043518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0008351-09f7-42d0-9290-5591f5b64dba None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.043597 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0008351-09f7-42d0-9290-5591f5b64dba None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.045044 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 363/718] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:33.049740 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab57466-57a3-4298-887e-822baa59fcc6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.050021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab57466-57a3-4298-887e-822baa59fcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.050291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab57466-57a3-4298-887e-822baa59fcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.069767 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3ab57466-57a3-4298-887e-822baa59fcc6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:33.070942 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ab57466-57a3-4298-887e-822baa59fcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:33.091220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 356/719] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:22:33.145216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:33.158795 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 364/720] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:33.429873 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:33.493720 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45a6b-c1ed-4317-b77c-d33d6397f090 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.494054 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45a6b-c1ed-4317-b77c-d33d6397f090 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.494272 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45a6b-c1ed-4317-b77c-d33d6397f090 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.495838 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 365/721] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:33.500945 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.501223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.501495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.520754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:33.521941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:33.538332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7eaf1-bd59-4d23-bc06-ee8efdcdce53 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:33.565679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:33.574463 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:33.575106 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.575409 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.575681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.576260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:33.577600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:33.587712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:33.588957 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:33.590070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 357/722] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:33.594600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:33.595923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 366/723] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:33.697011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172dd32-420e-43e1-a496-09345117ab2b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:33.731886 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:33.732487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.732761 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.733033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.733654 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:33.733950 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:33.735318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:33.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 358/724] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:33.748644 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:33.855600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bc570-44af-48f5-9547-9a69270a1eb0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:33.870675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:33.885218 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:33.893997 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:33.895139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:33.895450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:33.895719 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:33.896342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:33.899326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:33.899326 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:33.903668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 367/725] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:22:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:33.916509 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:33.954612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:34.006975 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d570b9-c055-4558-830a-1c42793eb7ff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:34.028889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70e286-2676-4b00-bc96-e6b02ff98e79 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:34.051169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:34.051845 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:34.052122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:34.052391 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] PATH_INFO: `/v3/services/image` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:34.052973 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:34.055376 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:34.066302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:34.067892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:34.068735 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:34.072932 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 359/726] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 05:22:33 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:34.073803 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:34.075203 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:34.076500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:34.076994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:34.077525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:34.078385 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:34.079987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:34.177630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a095938-a7b5-4f3f-a879-2377c4c5e67d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:34.202479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:34.203159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 368/727] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:22:34.229465 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:34.230013 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:34.230336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:34.230658 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:34.231340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] RBAC: Adding query filter params (name=image) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:34.231785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:34.233317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5cdec-d5e9-40dd-a189-9bece5d847ed None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:34.238954 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 360/728] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:34.245164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:34.300001 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:34.367097 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14d0b1-d909-414e-9567-f7b25563ce9f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:34.419725 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:34.420733 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:34.421236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:34.421702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:34.422537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:34.423048 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:34.424664 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182fb43b-b30b-4e6e-9714-73918df7c2cf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:34.430582 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 369/729] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:34.505300 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d7507-c5f6-4d8b-8833-4fd36dadaebf None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:34.505589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d7507-c5f6-4d8b-8833-4fd36dadaebf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:34.505746 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d7507-c5f6-4d8b-8833-4fd36dadaebf None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:34.508319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 361/730] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:34.517963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b336f-9379-48c9-aece-d5645bc6d741 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:34.517963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b336f-9379-48c9-aece-d5645bc6d741 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:34.517963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b336f-9379-48c9-aece-d5645bc6d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:34.535740 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9a7b336f-9379-48c9-aece-d5645bc6d741 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:34.537582 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9a7b336f-9379-48c9-aece-d5645bc6d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:34.614994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 370/731] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:34.884976 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d7507-c5f6-4d8b-8833-4fd36dadaebf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:34.996417 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d7507-c5f6-4d8b-8833-4fd36dadaebf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:35.031614 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c25f08-3f34-4975-b37c-626b5e902679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:35.032250 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c25f08-3f34-4975-b37c-626b5e902679 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:35.032520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c25f08-3f34-4975-b37c-626b5e902679 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:35.032746 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c25f08-3f34-4975-b37c-626b5e902679 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:35.033406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c25f08-3f34-4975-b37c-626b5e902679 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:35.034810 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c25f08-3f34-4975-b37c-626b5e902679 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:35.103174 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 362/732] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:34 2022] POST /identity/v3/services => generated 242 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:35.382448 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3a70b-45e7-4004-879e-3cee5d7db078 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:35.382983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3a70b-45e7-4004-879e-3cee5d7db078 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:35.383290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3a70b-45e7-4004-879e-3cee5d7db078 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:35.385069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 371/733] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:35.390339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:35.390731 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:35.391109 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:35.410922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:35.412413 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:35.447569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:35.492110 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 363/734] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:35.764005 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3a70b-45e7-4004-879e-3cee5d7db078 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:35.808927 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:35.868959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3a70b-45e7-4004-879e-3cee5d7db078 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:35.908260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:35.909247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:35.909763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:35.910790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:35.911882 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:35.913472 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:35.922611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:35.923637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:35.924500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:35.926891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 372/735] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:35.932872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:36.039459 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d85370-ea58-4bae-82a7-ea625fcb8acc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:36.076147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:36.076769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.077050 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.077323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.077972 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:36.078274 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:36.079666 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:36.085882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 364/736] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:36.091400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:36.201017 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e3896-f060-442f-af42-4574c08719a0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:36.204209 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:36.237127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:36.238001 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.238457 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.238878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.239701 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:36.240188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:36.241818 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:36.247548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 373/737] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:36.256598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:36.290795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:36.372046 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba9c88-acce-4cfa-8aca-bec2ab1a881d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:36.424053 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:36.424710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.424996 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.425271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.426025 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] RBAC: Adding query filter params (interface=admin, service_id=c98b0a24c18b4b4c804c82dbc7f95eb4, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:36.426339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:36.427742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a3cca-1b55-46a0-9451-7d07327ff39b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:36.439840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 365/738] 10.222.0.13 () {56 vars in 1249 bytes} [Mon Oct 17 05:22:36 2022] GET /identity/v3/endpoints?service_id=c98b0a24c18b4b4c804c82dbc7f95eb4&interface=admin®ion_id=RegionOne => generated 193 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:36.448395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:36.477601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b787b2-f0a3-4307-a208-310bcb796dff None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.477601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b787b2-f0a3-4307-a208-310bcb796dff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.477601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b787b2-f0a3-4307-a208-310bcb796dff None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.477601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 366/739] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:36.483281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb064-bb5b-44c3-bc29-bb89fd2af69b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.483549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb064-bb5b-44c3-bc29-bb89fd2af69b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.483820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb064-bb5b-44c3-bc29-bb89fd2af69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.501808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3c4fb064-bb5b-44c3-bc29-bb89fd2af69b None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:36.502989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3c4fb064-bb5b-44c3-bc29-bb89fd2af69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:36.573365 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a44070-1af6-4106-a115-c561ccace492 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:36.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6be752-3661-4ac9-90f9-25a6a0645701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:36.614432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be752-3661-4ac9-90f9-25a6a0645701 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.615223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be752-3661-4ac9-90f9-25a6a0645701 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.615535 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be752-3661-4ac9-90f9-25a6a0645701 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.616335 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6be752-3661-4ac9-90f9-25a6a0645701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:36.617767 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6be752-3661-4ac9-90f9-25a6a0645701 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:36.625739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 374/740] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:36 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:22:36.627648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 367/741] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:36.746903 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d39db-3818-4ac5-bc5d-f17f36e0599d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.747321 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d39db-3818-4ac5-bc5d-f17f36e0599d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.747573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d39db-3818-4ac5-bc5d-f17f36e0599d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.749109 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 375/742] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:36.755699 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:36.759236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:36.759236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:36.782388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:36.784687 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:36.811187 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:36.877818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 368/743] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:36.970281 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d39db-3818-4ac5-bc5d-f17f36e0599d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:37.011556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:37.086595 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d39db-3818-4ac5-bc5d-f17f36e0599d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:37.122441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:37.123053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.123381 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.123639 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.124279 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:37.125704 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:37.128372 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: WARNING keystone.server.flask.common [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 05:22:37.176750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.sql.core [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 05:22:37.176750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:22:37.176750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [parameters: {'id': '6f85573f43134a4bb02ccbee6be6e4ce', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 05:22:37.176750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6f85573f43134a4bb02ccbee6be6e4ce.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6f85573f43134a4bb02ccbee6be6e4ce. Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:22:37.184667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:22:37.185467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application yield resource Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise self.value Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application session.commit() Oct 17 05:22:37.186455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:22:37.188350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:22:37.190471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:22:37.192527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [parameters: {'id': '6f85573f43134a4bb02ccbee6be6e4ce', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:37.194035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:37.197061 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6f85573f43134a4bb02ccbee6be6e4ce. Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:37.201763 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 376/744] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Oct 17 05:22:36 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:37.203028 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:37.203536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:37.294222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:37.307724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1011b-b58e-425e-bf3e-726f9d2b5591 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:37.308133 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5dfb4-651e-4465-828d-29e77b5a096c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:37.350523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741f36-a378-471b-b5d2-faac88da81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:37.351278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:37.351783 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741f36-a378-471b-b5d2-faac88da81e9 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.351989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.352229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.352565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.352897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741f36-a378-471b-b5d2-faac88da81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.353364 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741f36-a378-471b-b5d2-faac88da81e9 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.354221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7741f36-a378-471b-b5d2-faac88da81e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:37.356665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7741f36-a378-471b-b5d2-faac88da81e9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:37.360510 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:37.363195 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:37.374891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:37.380228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:37.381142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 369/745] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:37.382964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:37.399917 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:37.453085 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 377/746] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:37 2022] POST /identity/v3/services => generated 242 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:37.454394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b107-4fe5-4055-ae79-a466beead9ea None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.454677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b107-4fe5-4055-ae79-a466beead9ea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.454944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b107-4fe5-4055-ae79-a466beead9ea None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.456320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 378/747] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:37.467342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.467342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.467342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.491232 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:37.493130 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:37.503792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417069-737d-4689-8b50-4f621c888608 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:37.552592 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:37.553534 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.553994 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.554469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.555520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:37.556010 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:37.557642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:37.577348 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 370/748] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:37.595507 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 379/749] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:37 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:37.858837 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:37.929396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:37.960667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39b43-ad29-47fb-9cd7-4878b503b84a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:37.994900 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:37.995557 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:37.995829 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:37.996096 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:37.996663 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:37.998034 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:38.007576 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:38.008645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:38.009512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 371/750] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:38.014885 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:38.093698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:38.116196 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114770-dfdb-47ec-9d4f-c0d223c09ce2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:38.151586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:38.152197 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:38.152474 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:38.152746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:38.153428 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:38.153732 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:38.155094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:38.160642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 380/751] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:38.166464 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:38.272558 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8329bd-bec8-49d8-a5e0-f51104b1fa3a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:38.307150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:38.307833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:38.308112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:38.308381 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:38.308995 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:38.309350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:38.310711 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:38.311940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:38.316881 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 372/752] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:38.327432 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:38.434774 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525e87f-297e-4df1-ac95-6d54ec9b1cb2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:38.470203 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:38.471496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:38.471939 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:38.472351 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:38.473680 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] RBAC: Adding query filter params (interface=public, service_id=9efa604cf6c843b9ad6dcdec2ba8b6df, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:38.474139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:38.475661 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da3d4d-f80d-4b87-b73a-a49be0333026 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:38.481333 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 381/753] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:38 2022] GET /identity/v3/endpoints?service_id=9efa604cf6c843b9ad6dcdec2ba8b6df&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:38.487031 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:38.593157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b958b96-96b7-460c-94a8-493d68778472 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:38.632220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9fc0-ab0c-44e6-9ae6-9699cb2d0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:38.633123 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9fc0-ab0c-44e6-9ae6-9699cb2d0b43 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:38.633582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9fc0-ab0c-44e6-9ae6-9699cb2d0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:38.634002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9fc0-ab0c-44e6-9ae6-9699cb2d0b43 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:38.634733 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f9fc0-ab0c-44e6-9ae6-9699cb2d0b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:38.636241 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f9fc0-ab0c-44e6-9ae6-9699cb2d0b43 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:38.643350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 373/754] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:38 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:22:39.026340 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad387fff-d4f6-49ba-bdd0-9bff4e767701 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.026802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad387fff-d4f6-49ba-bdd0-9bff4e767701 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.027197 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad387fff-d4f6-49ba-bdd0-9bff4e767701 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.028913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 382/755] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:39.034155 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.034547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.034918 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.053963 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:39.055145 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:39.113872 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:39.124917 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 374/756] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:39 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:39.392324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad387fff-d4f6-49ba-bdd0-9bff4e767701 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:39.401669 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:39.504395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad387fff-d4f6-49ba-bdd0-9bff4e767701 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:39.540714 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:39.541349 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.541627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.541897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.542475 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:39.543870 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fb423e1e-f10b-4217-9d37-14d9827413b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:39.554072 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:39.555152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:22:39.556059 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:39.559021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 383/757] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:22:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:39.562095 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:39.669553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2863d-4728-453b-a61b-a2336ea6fdf3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:39.698698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9d30c-c554-4ca0-beea-0a3d4fcbf9a8 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.699022 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9d30c-c554-4ca0-beea-0a3d4fcbf9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.699256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9d30c-c554-4ca0-beea-0a3d4fcbf9a8 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.701042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 384/758] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:39.707330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:39.707473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.707553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.707625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.707716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.707789 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.707859 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.707859 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:39.708158 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:39.709455 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:39.716219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 375/759] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:39.721922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:39.726781 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:39.728110 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:39.787746 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:39.810621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:39.817411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 385/760] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:39 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:39.836247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782d6c-985a-4c33-9dbd-3ecb63be0877 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:39.877711 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:39.878601 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.879083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.879528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.880296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:39.880744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:39.882297 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e21920-8d50-4755-bdfe-a003ce2163c8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:39.887668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 376/761] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:22:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:39.893501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:39.956223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5548a-6e49-4d42-bda3-7a9ecaa874d5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.957564 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5548a-6e49-4d42-bda3-7a9ecaa874d5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.957995 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5548a-6e49-4d42-bda3-7a9ecaa874d5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.960459 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 377/762] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:22:39.965048 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:39.965330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:39.965603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:39.987539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:39.988928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:40.005435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e25d1-8a14-4a8b-9da1-da9e3dd01064 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.042911 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.043882 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.044378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.044860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.045770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.047334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.070588 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 378/763] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:39 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:40.102372 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.140216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 386/764] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:40.160609 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:40.221150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d8a8d-963e-4ed5-ab09-5fe153bf3ec4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.266940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.267633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.267916 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.268177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] PATH_INFO: `/v3/services/image` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.268840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.270339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:40.280267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:40.281331 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:40.282176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.286673 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 379/765] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:40.290186 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.357197 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:40.377875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.405526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5d717-9868-494f-a52c-210bd473bdbb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.449702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.450315 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.450837 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.451113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.452167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:40.452734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.454384 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.464534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 387/766] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:40.472695 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.499046 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6c1c0-86d3-4e3e-bdf6-c7babcd5837c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.510324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:40.540706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.541538 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.541881 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.542219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.542946 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.544743 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.548600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: WARNING keystone.server.flask.common [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 05:22:40.586907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 05:22:40.586907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 05:22:40.586907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [parameters: {'user_id': '881548f6b9f54b0985c30bbb670809dc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 05:22:40.586907 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:22:40.594782 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b156-c0d8-4122-b4f2-af1884fbfe4a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 05:22:40.608525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:22:40.609706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:22:40.610828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:22:40.611929 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:22:40.613801 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:22:40.614956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'user_id': '881548f6b9f54b0985c30bbb670809dc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:40.616171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 05:22:40.617210 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 380/767] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:22:40 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:40.619630 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.633304 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:40.645103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.646079 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.646564 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.647110 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.648020 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:40.648565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.650271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.661273 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 388/768] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:40.662811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.740433 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbb49e-b26f-41ed-b8d0-7990d53f08fe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.785759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.792071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.792071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.792071 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.798390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.799693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4faa0d-6dcb-4508-ab3e-19d06722d820 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.800072 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:40.818226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:40.819233 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:40.820958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 381/769] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:40.828680 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.848344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.849103 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.849452 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.849791 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.850565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] RBAC: Adding query filter params (interface=public, service_id=a5fcca334b574e52835d2296a4a77470, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:40.850937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.852412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340bf43-8756-4c83-9428-c49526f3d465 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:40.861387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 389/770] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/endpoints?service_id=a5fcca334b574e52835d2296a4a77470&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:40.863093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7204d8-36f4-44a9-bd0e-d050380fd5c1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.863317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7204d8-36f4-44a9-bd0e-d050380fd5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.863612 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7204d8-36f4-44a9-bd0e-d050380fd5c1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.864976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 390/771] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:40.865537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7204d8-36f4-44a9-bd0e-d050380fd5c1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:40.872119 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:40.948878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf7eb7-585a-47d3-93fe-1c083c05ecfe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:40.986755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:40.987401 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:40.987684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:40.987956 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:40.988656 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:40.988958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:40.990375 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13840145-4ef5-4cec-800d-6cd967c27db2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:41.028997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 382/772] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:41.030120 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:41.030409 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:41.030685 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:41.050111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:41.051755 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:41.052129 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:41.130536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7204d8-36f4-44a9-bd0e-d050380fd5c1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:41.132963 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 383/773] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:41 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:41.170937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:41.172034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:41.172567 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:41.173046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:41.173890 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:41.176215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:41.192004 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 391/774] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:22:40 2022] GET /identity/v3/services => generated 1957 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:22:41.405248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:41.428400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:41.506271 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdaa6e-a0fe-4301-89d2-4f5e54f53062 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:41.540669 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:41.541304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:41.541577 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:41.541843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:41.542416 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:41.543804 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:41.553269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:41.554237 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:41.555083 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 384/775] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:41.561205 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:41.659176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91069aa3-edf7-4589-90ad-1467ebd0da7e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:41.676658 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:41.693784 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:41.694386 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:41.694662 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:41.694928 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:41.695562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:41.695857 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:41.697201 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:41.706578 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 392/776] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:41.707440 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:41.790582 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:41.806768 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54905a-0eb8-4b72-baec-d6448730f04c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:41.840093 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:41.840840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:41.841223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:41.841587 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:41.842302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:41.842718 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:41.844175 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0324b-e2ee-4add-af15-1b824a6c0a2f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:41.850425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 385/777] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:22:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:41.855928 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:41.956549 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5609cd-d717-4be1-a9d4-d081eb7689a7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:41.990527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f39be9-9d99-4a92-8494-3f25ab985cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:41.991264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f39be9-9d99-4a92-8494-3f25ab985cef None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:41.991663 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f39be9-9d99-4a92-8494-3f25ab985cef None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:41.992030 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f39be9-9d99-4a92-8494-3f25ab985cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:41.992794 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f39be9-9d99-4a92-8494-3f25ab985cef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:41.994243 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f39be9-9d99-4a92-8494-3f25ab985cef None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:42.114976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 393/778] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:22:42.432621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85c0e9-b0d2-4560-88ef-dc1c4d31c920 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:42.432935 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85c0e9-b0d2-4560-88ef-dc1c4d31c920 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:42.433192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85c0e9-b0d2-4560-88ef-dc1c4d31c920 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:42.434683 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 386/779] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:42.440031 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:42.440322 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:42.440600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:42.459978 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:42.461186 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:42.489696 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:42.543597 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 394/780] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:42.806253 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85c0e9-b0d2-4560-88ef-dc1c4d31c920 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:42.911269 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85c0e9-b0d2-4560-88ef-dc1c4d31c920 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:42.945402 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a54240-e497-4d06-8f6a-e1d12739f464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:42.946059 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a54240-e497-4d06-8f6a-e1d12739f464 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:42.946363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a54240-e497-4d06-8f6a-e1d12739f464 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:42.946642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a54240-e497-4d06-8f6a-e1d12739f464 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:42.947793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a54240-e497-4d06-8f6a-e1d12739f464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:42.949228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a54240-e497-4d06-8f6a-e1d12739f464 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:42.950794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 387/781] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:42.956703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:43.056676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f961a56-4d8c-45ce-8557-d3ffd8319c1b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:43.092029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd07f2f-0e0c-4318-8986-3432e704dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:43.092638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07f2f-0e0c-4318-8986-3432e704dcce None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:43.092919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07f2f-0e0c-4318-8986-3432e704dcce None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:43.093189 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07f2f-0e0c-4318-8986-3432e704dcce None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:43.093843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd07f2f-0e0c-4318-8986-3432e704dcce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:43.095290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd07f2f-0e0c-4318-8986-3432e704dcce None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:43.223344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 395/782] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:22:42 2022] POST /identity/v3/projects => generated 311 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:43.365523 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9902f-1d78-4d4b-ab62-2bc1bf2c5e1b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:43.365910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9902f-1d78-4d4b-ab62-2bc1bf2c5e1b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:43.366086 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9902f-1d78-4d4b-ab62-2bc1bf2c5e1b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:43.367690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 388/783] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:43.373097 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:43.373380 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:43.373660 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:43.392473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:43.393654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:43.404008 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:43.466245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 396/784] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:43.731559 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9902f-1d78-4d4b-ab62-2bc1bf2c5e1b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:43.751615 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:43.835460 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9902f-1d78-4d4b-ab62-2bc1bf2c5e1b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:43.868741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:43.869680 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:43.870141 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:43.870590 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] PATH_INFO: `/v3/services/image` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:43.871363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:43.872970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:43.882355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:43.883462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:43.887393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 389/785] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 05:22:43 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:43.890167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:43.989998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f066993-14c7-4457-8634-36cbbf57d8eb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:44.024044 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:44.024646 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:44.024920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:44.025194 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:44.025811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:44.026107 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:44.027485 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:44.032370 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 397/786] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:44.038120 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:44.117955 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:44.137064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d098ad-e78d-4763-bbbe-db2a19839655 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:44.172828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:44.173432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:44.173707 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:44.173975 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:44.174589 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] RBAC: Adding query filter params (type=image) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:44.174888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:44.176811 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3ba6b-0f98-4e99-8d34-3565daf4d13a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:44.181674 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 390/787] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:44.187115 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:44.198679 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:44.289518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c7acb-cb6b-4ffb-982a-dbcb1a0a7937 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:44.322363 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:44.322955 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:44.323230 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:44.323527 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:44.324186 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:44.325531 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:44.349648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a714f-1fda-493a-8301-4d3439f11a3e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:44.349933 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a714f-1fda-493a-8301-4d3439f11a3e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:44.350209 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a714f-1fda-493a-8301-4d3439f11a3e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:44.351624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 391/788] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:44.356276 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:44.356581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:44.356851 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:44.375782 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:44.376980 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:44.424264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 398/789] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:22:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:22:44.458043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 392/790] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:44.725012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:44.788396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:44.826485 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e9c3-baa6-45d7-9eff-d154dd8f0e25 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:44.860808 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:44.861440 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:44.861719 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:44.861989 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:44.872987 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:44.878094 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:44.879056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:44.880622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:44.883378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 399/791] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:22:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:44.885196 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:44.958107 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:44.987602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ee794-b133-4582-99c4-48f49b28b000 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.021146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.021751 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.022025 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.022291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.022938 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:45.023236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.024638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.038039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 393/792] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:22:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:45.039447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.136311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea66dae-9f4b-4cb1-9fe8-af617030ed1c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.159153 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:45.170282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.170913 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.172098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.172098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.182195 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.183657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:45.198259 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:45.199226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:45.200164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:45.200164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:45.200164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:45.200164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 400/793] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:45.206440 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.308243 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c583190-8354-4697-bead-b44cf1ba23a4 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.341625 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:45.347885 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.348631 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.348987 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.349437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.350319 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:45.350706 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.352217 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2272e39-fc27-4caf-97f6-d8f9725a9749 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.365345 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 394/794] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:45.370812 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.447080 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8cf47-2885-4779-9af4-fdfb21aa9ca0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.447420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8cf47-2885-4779-9af4-fdfb21aa9ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.447673 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8cf47-2885-4779-9af4-fdfb21aa9ca0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.449178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 395/795] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:45.453914 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.454200 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.454479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.473434 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:45.474623 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:45.481667 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e7bd3-c0f4-4538-ba4e-2b9dd369486f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.517923 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.518608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.518892 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.519218 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.527836 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.529295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:45.541221 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:45.542156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:45.543113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 401/796] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:45.547888 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.554979 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 396/797] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:45.586785 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:45.652080 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a86a4-3513-4309-b582-0d768b0c54e8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.686636 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.687242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.687549 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.687825 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.688541 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:45.688867 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.690295 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.708354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 402/798] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:45.716354 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.812406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20900f5d-d52d-4d73-a5e9-f50fcb3d5fa7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.824334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.837382 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:45.848934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.849575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.849855 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.850127 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.861285 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.863600 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:45.878547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:45.880205 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:45.881149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:45.881149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:45.881149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:45.881149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 397/799] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:45.885542 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.924175 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:45.931799 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39251-a0e3-4e88-947d-5a7835e77354 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:45.966999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:45.967676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:45.967952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:45.968245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:45.969323 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:45.970746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:45.977404 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 403/800] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:45.978282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:45.998887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649622e-61e5-4c95-92fe-ec0211784511 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:46.038927 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:46.039611 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.039886 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.040160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.040780 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:46.041088 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:46.042431 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:46.054820 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 398/801] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:46.060407 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:46.080385 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811586a-b083-47d7-b233-7627802c7258 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:46.094817 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:46.119465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:46.120066 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.120347 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.120623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.121255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:46.122700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:46.165391 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0db80-793d-4d0b-a669-5a6f7a430f6c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:46.198362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:46.198981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.199254 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.199549 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.205921 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:46.207380 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:46.218613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:46.219520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:46.220468 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:46.225206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 399/802] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:46.225614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 404/803] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:22:45 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:22:46.225964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:46.334324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:46.342506 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d030316-75b9-4e58-9a03-788d3ce0ff53 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:46.379818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:46.380349 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.380623 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.380891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.381587 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:46.381891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:46.383355 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa34e1c-94aa-4927-a45a-55e9cef7718a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:46.397149 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 400/804] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:46.402830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:46.427861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:46.511611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883599f4-cf91-46b3-be53-11c8c1f1a1d3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:46.550044 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cc6b6-023b-4748-b450-8dc98e90a280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:46.550741 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cc6b6-023b-4748-b450-8dc98e90a280 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.551022 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cc6b6-023b-4748-b450-8dc98e90a280 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.551324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cc6b6-023b-4748-b450-8dc98e90a280 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/231350e78b2740a4b0aef761f7baa4e6/roles/40bd8db9aded43d38ac338975828fc90` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.553440 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cc6b6-023b-4748-b450-8dc98e90a280 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=231350e78b2740a4b0aef761f7baa4e6, role_id=40bd8db9aded43d38ac338975828fc90)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:46.554873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cc6b6-023b-4748-b450-8dc98e90a280 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:46.610441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 405/805] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:46 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/231350e78b2740a4b0aef761f7baa4e6/roles/40bd8db9aded43d38ac338975828fc90 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:22:46.627684 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e42a-cbbf-4e57-b21b-1e35e951e06d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.627996 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e42a-cbbf-4e57-b21b-1e35e951e06d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.628304 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1e42a-cbbf-4e57-b21b-1e35e951e06d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.629894 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 401/806] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:46.635774 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:46.636082 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:46.636352 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:46.656665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:46.657873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:46.736375 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 406/807] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:46.997816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e42a-cbbf-4e57-b21b-1e35e951e06d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:47.099042 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1e42a-cbbf-4e57-b21b-1e35e951e06d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:47.133021 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:47.133629 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:47.133905 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:47.134172 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:47.134845 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:47.136298 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:47.169553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:22:47.169553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:22:47.169553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [parameters: {'id': '69a0ea9409c24b8ebf0f39e276605243', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:22:47.169553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:22:47.177498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:22:47.178320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:22:47.179272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:22:47.180201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:22:47.181085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:22:47.181858 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'id': '69a0ea9409c24b8ebf0f39e276605243', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:22:47.182725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:47.185281 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:47.187396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 402/808] 10.222.0.13 () {60 vars in 1137 bytes} [Mon Oct 17 05:22:46 2022] POST /identity/v3/domains => generated 190 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:47.189117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:47.198362 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:47.284609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1642-6daa-4750-a4f8-5408567acd47 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:47.320499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:47.321219 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:47.321563 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:47.321898 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:47.332187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:47.333697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7745da5e-61c7-4e0b-bcc8-82a9f0b23f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:47.348239 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:47.349382 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:47.350518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:47.350518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:47.350518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:47.351450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 407/809] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:47.356928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:47.378330 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:47.467669 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cce2b-08ad-44ec-9233-8d2709bbc1f1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:47.504838 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:47.505699 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:47.506122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:47.506568 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:47.507383 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:47.507877 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:47.509369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b36d4-b0e3-441b-bd3f-4d4091d5ad1e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:47.521830 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 403/810] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:48.408746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a6414-7865-4609-818d-d1fecbe21b7a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:48.409308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a6414-7865-4609-818d-d1fecbe21b7a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:48.409308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a6414-7865-4609-818d-d1fecbe21b7a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:48.413125 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 408/811] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:48.416337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:48.416772 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:48.416895 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:48.434499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:48.435692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:48.499611 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 404/812] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:48.762356 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027620b-e339-4a7c-ab89-363098677257 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:48.762621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027620b-e339-4a7c-ab89-363098677257 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:48.762936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027620b-e339-4a7c-ab89-363098677257 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:48.764289 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 409/813] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:48.768095 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:48.769620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:48.769901 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:48.770172 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:48.779219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:48.788518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:48.789737 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:48.844044 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:48.855556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 410/814] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:22:48.867640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abff0c-512b-4753-9c5c-54c4d1874b02 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:48.900318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:48.900964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:48.901239 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:48.901506 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:48.911896 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:48.916520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:48.917457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:48.918443 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 405/815] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:48.923624 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.019835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a923b7e-4563-4225-90a7-7d539d4bd9e6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.051572 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.052159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.052434 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.052700 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.053342 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:49.053633 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.054971 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.062627 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 411/816] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:49.068866 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.123026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.134687 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:49.167977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275769b-f888-4f18-9438-30e26bc8ff97 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.177389 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:49.202247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.202867 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.203146 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.203450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] PATH_INFO: `/v3/users/24a8f1e174884743baa6b2a29a79bc03` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.216793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] RBAC: Authorizing `identity:get_user(user_id=24a8f1e174884743baa6b2a29a79bc03)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.218213 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.219836 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 406/817] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/users/24a8f1e174884743baa6b2a29a79bc03 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:49.223525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc7880-13bb-48d2-9ccd-3997e1087991 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.225694 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.256414 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.257031 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.257311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.257580 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.267972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:49.273743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:49.274672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:49.275565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 412/818] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:49.280539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.321078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9916e-d805-4811-ac6d-4caed6a86b67 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.346422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:49.355188 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.355826 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.356100 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.356521 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] PATH_INFO: `/v3/projects/1bcea60d87c5481abbf565c218161a9b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.357979 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] RBAC: Authorizing `identity:get_project(project_id=1bcea60d87c5481abbf565c218161a9b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.359581 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.361176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 407/819] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/projects/1bcea60d87c5481abbf565c218161a9b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:49.367536 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.374986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203e93-d710-4461-84f5-e7be808833cf None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.388164 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:49.408790 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.410025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.410025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.410025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.410686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:49.410983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.412628 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.421194 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 413/820] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:22:49.427460 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.467055 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583c737-628d-49f2-8742-e9fbe4524c22 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.502219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.502870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.503147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.503987 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] PATH_INFO: `/v3/projects/1bcea60d87c5481abbf565c218161a9b/users/24a8f1e174884743baa6b2a29a79bc03/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.505852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bcea60d87c5481abbf565c218161a9b, user_id=24a8f1e174884743baa6b2a29a79bc03, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.507262 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.524718 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e316d3-93a3-4b26-b50a-23f1a41f84ed None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.557262 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.557869 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.558147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.558416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.568573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.570004 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.575678 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 408/821] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:49 2022] PUT /identity/v3/projects/1bcea60d87c5481abbf565c218161a9b/users/24a8f1e174884743baa6b2a29a79bc03/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:49.585777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:49.587021 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:49.587982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:49.587982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:49.587982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:49.587982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 414/822] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:49.593189 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.651186 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:49.698690 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db28f2-835a-4420-8135-da43f404f8e9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.734403 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.735011 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.735285 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.735578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.737037 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:49.737037 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.738404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.750268 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 409/823] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:49.756276 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:49.788831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:49.854705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650497-8242-4c35-b6a1-f366dd68339e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:49.887119 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:49.887760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:49.888035 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:49.888302 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:49.895114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:49.896580 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:22:49.907562 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:49.908677 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:49.910217 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 415/824] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:49.915085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:50.010378 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c658ba-9b66-45fc-a963-e87f9732a96f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:50.023425 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:50.043390 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:50.043921 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:50.044196 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:50.044465 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:50.045177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:50.045477 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:50.046878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:50.064611 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 410/825] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:22:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:50.070306 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:50.151456 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:50.165760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b9d67-0c0e-4bbf-91e3-5ea185485f89 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:50.197710 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:50.198311 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:50.198589 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:50.198857 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:50.262708 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:50.264178 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:50.343640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:50.358906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 416/826] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:22:50.364731 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:50.465396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce2ea-db57-4a8c-8fcc-4632b0c05222 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:50.498104 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:50.498713 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:50.498983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:50.499252 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:50.509207 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:50.510613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:22:50.524971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:50.526043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:22:50.526985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:22:50.526985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:22:50.526985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:50.526985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 411/827] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:22:50.531728 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:50.625513 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3af550-feb5-4285-817c-0c124e3b08e1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:50.658301 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:50.658893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:50.659173 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:50.659479 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:50.660128 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:50.660429 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:50.662045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:50.672880 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 417/828] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:50.679225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:50.704303 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:50.776467 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496655-0d52-4d25-80c9-734c9d822e85 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:50.923639 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:50.924167 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:50.924439 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:50.924712 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:50.931539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:50.932936 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:22:50.944135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:50.945151 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:22:50.946160 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 412/829] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:50.951047 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:50.995660 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:51.046413 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68509fcf-d632-4512-85fd-34ef9f900bba None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:51.079130 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:51.079755 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:51.080029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:51.080296 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:51.080968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:51.081264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:51.082704 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78fb8d-e379-4632-8ef5-2780f34680bc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:51.094100 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 418/830] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:22:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:51.100041 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:51.179990 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:51.195317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09f0ea-f3fa-422c-9370-7a4073434ec9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:51.228014 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:51.228613 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:51.228887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:51.229157 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:51.229869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] RBAC: Adding query filter params (role.id=40bd8db9aded43d38ac338975828fc90, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=231350e78b2740a4b0aef761f7baa4e6) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:51.230174 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:51.231603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3886d-8f31-45a1-8f5b-3c5c19b0dd62 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:51.237144 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 413/831] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:51 2022] GET /identity/v3/role_assignments?user.id=231350e78b2740a4b0aef761f7baa4e6&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=40bd8db9aded43d38ac338975828fc90 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:51.712091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8acd28-bbc5-493e-a4cf-4ecc305c5165 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:51.712609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8acd28-bbc5-493e-a4cf-4ecc305c5165 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:51.712609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8acd28-bbc5-493e-a4cf-4ecc305c5165 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:51.713968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 419/832] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:51.719498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:51.719931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:51.720051 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:51.738692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:51.739915 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:51.806491 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 414/833] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:52.076257 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8acd28-bbc5-493e-a4cf-4ecc305c5165 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:52.176521 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8acd28-bbc5-493e-a4cf-4ecc305c5165 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:52.184085 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:52.211496 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:52.212203 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:52.212461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:52.212733 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:52.223703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:22:52.228779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:22:52.229950 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:22:52.231226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 420/834] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:22:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:22:52.236282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:52.313758 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:52.335356 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf5c71-37ef-4f9f-824e-af2deb2b78bb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:52.368704 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:52.369393 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:52.369676 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:52.370028 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:52.370865 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:52.371199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:52.372710 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:52.380889 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 415/835] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:22:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:52.386678 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:52.481378 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54595bd-fd28-447e-bebb-21862f9adadc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:52.513135 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:52.513746 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:52.514026 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:52.514296 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] PATH_INFO: `/v3/users/24a8f1e174884743baa6b2a29a79bc03` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:52.515528 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=24a8f1e174884743baa6b2a29a79bc03)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:52.516955 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:52.518702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 421/836] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:22:52 2022] GET /identity/v3/users/24a8f1e174884743baa6b2a29a79bc03 => generated 285 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:52.524796 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:52.619878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337faff-c266-40c5-a502-4c6d988b77cb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:52.651551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:52.652142 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:52.652420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:52.652691 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:52.663337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:52.717092 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:52.718499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:52.810857 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 416/837] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:22:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:22:52.817335 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:52.849685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:52.913193 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db10440-3a16-4c9e-92bd-5dbe65cac7e4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:52.945367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791e735-616b-4356-a367-9d8b78db04fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:52.945972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791e735-616b-4356-a367-9d8b78db04fd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:52.946249 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791e735-616b-4356-a367-9d8b78db04fd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:52.946518 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791e735-616b-4356-a367-9d8b78db04fd None admin] PATH_INFO: `/v3/projects/1bcea60d87c5481abbf565c218161a9b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:52.947637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791e735-616b-4356-a367-9d8b78db04fd None admin] RBAC: Authorizing `identity:get_project(project_id=1bcea60d87c5481abbf565c218161a9b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:52.949056 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791e735-616b-4356-a367-9d8b78db04fd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:52.950634 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 422/838] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:22:52 2022] GET /identity/v3/projects/1bcea60d87c5481abbf565c218161a9b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:52.957180 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:52.996360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:53.051225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686b81e-9de1-4bc5-843f-752f0dd79063 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:53.084194 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:53.084779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:53.085055 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:53.085322 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:53.086025 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=1bcea60d87c5481abbf565c218161a9b, user.id=24a8f1e174884743baa6b2a29a79bc03) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:22:53.086329 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:53.087756 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c2f3c-f994-44ef-bba3-00e22336ef8a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:53.093232 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 417/839] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:22:52 2022] GET /identity/v3/role_assignments?user.id=24a8f1e174884743baa6b2a29a79bc03&scope.project.id=1bcea60d87c5481abbf565c218161a9b&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:55.208674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70c919-634a-4eec-b182-d4025048f264 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:55.209242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70c919-634a-4eec-b182-d4025048f264 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:55.209242 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70c919-634a-4eec-b182-d4025048f264 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:55.210548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 423/840] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:55.215816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:55.216207 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:55.216366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:55.234364 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:55.235566 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:55.299351 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 418/841] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:55.566530 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70c919-634a-4eec-b182-d4025048f264 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:55.663964 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70c919-634a-4eec-b182-d4025048f264 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:55.665431 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:55.696634 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fcf80-2aa7-4e14-8545-76182b1897ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:55.697243 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fcf80-2aa7-4e14-8545-76182b1897ef None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:55.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fcf80-2aa7-4e14-8545-76182b1897ef None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:55.697800 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fcf80-2aa7-4e14-8545-76182b1897ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:55.698997 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fcf80-2aa7-4e14-8545-76182b1897ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:55.700536 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fcf80-2aa7-4e14-8545-76182b1897ef None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:55.702052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 424/842] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:22:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:55.708359 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:55.739516 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:55.801714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326398-6348-40ba-9573-e43a6dbaa884 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:55.833593 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45e97d-bd0c-48c6-ad65-75251e410a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:55.834126 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e97d-bd0c-48c6-ad65-75251e410a0c None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:55.834405 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e97d-bd0c-48c6-ad65-75251e410a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:55.834672 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e97d-bd0c-48c6-ad65-75251e410a0c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:55.835324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45e97d-bd0c-48c6-ad65-75251e410a0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:55.836725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45e97d-bd0c-48c6-ad65-75251e410a0c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:55.911814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 419/843] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:22:55 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:22:58.050622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8598e3-a96e-453a-b375-5e11bd72255b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:58.051157 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8598e3-a96e-453a-b375-5e11bd72255b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:58.051157 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8598e3-a96e-453a-b375-5e11bd72255b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:58.052505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 425/844] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:22:58.057665 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:58.058064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:58.058213 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:58.076264 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:22:58.077437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:22:58.140122 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 420/845] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:22:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:22:58.414606 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8598e3-a96e-453a-b375-5e11bd72255b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:58.513186 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8598e3-a96e-453a-b375-5e11bd72255b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:58.545297 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:58.545905 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:58.546187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:58.546461 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] PATH_INFO: `/v3/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:58.547853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] RBAC: Authorizing `identity:get_role(role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:58.549209 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:58.550998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 426/846] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:22:58 2022] GET /identity/v3/roles/517b2b1aaac34e92b002aec2385431d0 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:58.556679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:58.577503 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:58.652257 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b375bd4-f062-49c5-9342-2ab34a4af871 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:58.684504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:58.685066 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:58.685346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:58.685642 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] PATH_INFO: `/v3/users/e11c7496e35447a1aea9774c2980130e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:58.699185 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=e11c7496e35447a1aea9774c2980130e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:58.700629 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:58.702182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 421/847] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:22:58 2022] GET /identity/v3/users/e11c7496e35447a1aea9774c2980130e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:58.707970 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:58.746665 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:58.859603 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bded-81de-4a30-8567-cb64bb8c1ff7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:58.892312 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099682dc-b6c4-4fd0-a904-73ba1cd66d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:58.892931 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099682dc-b6c4-4fd0-a904-73ba1cd66d80 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:58.893208 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099682dc-b6c4-4fd0-a904-73ba1cd66d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:58.893503 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099682dc-b6c4-4fd0-a904-73ba1cd66d80 None admin] PATH_INFO: `/v3/projects/1bcea60d87c5481abbf565c218161a9b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:58.895159 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099682dc-b6c4-4fd0-a904-73ba1cd66d80 None admin] RBAC: Authorizing `identity:get_project(project_id=1bcea60d87c5481abbf565c218161a9b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:58.896607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099682dc-b6c4-4fd0-a904-73ba1cd66d80 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:58.898223 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 427/848] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:22:58 2022] GET /identity/v3/projects/1bcea60d87c5481abbf565c218161a9b => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:22:58.904204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:22:58.978047 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:22:58.997791 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e768014-d338-4c41-8225-859b2fff8ea8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:22:59.031284 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628708f1-139c-48bb-97c2-406a1b782a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:22:59.031942 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628708f1-139c-48bb-97c2-406a1b782a5a None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:22:59.032213 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628708f1-139c-48bb-97c2-406a1b782a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:22:59.032480 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628708f1-139c-48bb-97c2-406a1b782a5a None admin] PATH_INFO: `/v3/projects/1bcea60d87c5481abbf565c218161a9b/users/e11c7496e35447a1aea9774c2980130e/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:22:59.034868 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628708f1-139c-48bb-97c2-406a1b782a5a None admin] RBAC: Authorizing `identity:create_grant(project_id=1bcea60d87c5481abbf565c218161a9b, user_id=e11c7496e35447a1aea9774c2980130e, role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:22:59.036544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628708f1-139c-48bb-97c2-406a1b782a5a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:22:59.106153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 422/849] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:22:58 2022] PUT /identity/v3/projects/1bcea60d87c5481abbf565c218161a9b/users/e11c7496e35447a1aea9774c2980130e/roles/517b2b1aaac34e92b002aec2385431d0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:23:01.213463 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864433cf-b6fc-4d23-9f31-1f17e95e086a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:01.213993 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864433cf-b6fc-4d23-9f31-1f17e95e086a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:01.214080 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864433cf-b6fc-4d23-9f31-1f17e95e086a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:01.215412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 428/850] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:01.220687 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:01.221134 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:01.221225 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:01.240169 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:01.241350 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:01.305344 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 423/851] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:01.571692 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864433cf-b6fc-4d23-9f31-1f17e95e086a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:01.668238 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864433cf-b6fc-4d23-9f31-1f17e95e086a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:01.675173 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:01.700459 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:01.701069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:01.701344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:01.701611 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] PATH_INFO: `/v3/roles/517b2b1aaac34e92b002aec2385431d0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:01.702948 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] RBAC: Authorizing `identity:get_role(role_id=517b2b1aaac34e92b002aec2385431d0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:01.704324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:01.705601 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 429/852] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:23:01 2022] GET /identity/v3/roles/517b2b1aaac34e92b002aec2385431d0 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:01.711966 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:01.774968 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:01.806168 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf58a8a-27f3-4aaa-bef8-75102a6b3fe0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:01.839552 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:01.841732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:01.841732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:01.841732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] PATH_INFO: `/v3/users/e11c7496e35447a1aea9774c2980130e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:01.841989 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] RBAC: Authorizing `identity:get_user(user_id=e11c7496e35447a1aea9774c2980130e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:01.843244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:01.844811 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 424/853] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:23:01 2022] GET /identity/v3/users/e11c7496e35447a1aea9774c2980130e => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:01.850646 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:01.946588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c914e1-1a5b-4129-840c-f9595776a8ac None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:01.978975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:01.979590 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:01.979881 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:01.980157 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:02.020482 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:02.044149 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:02.045580 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:02.139967 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 430/854] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:23:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:23:02.145882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:02.240586 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed393-b419-4af5-9c35-f355b73762fe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:02.272924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cfb2f-7746-43c4-8ff9-8acb38faa675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:02.273534 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cfb2f-7746-43c4-8ff9-8acb38faa675 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:02.273814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cfb2f-7746-43c4-8ff9-8acb38faa675 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:02.274087 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cfb2f-7746-43c4-8ff9-8acb38faa675 None admin] PATH_INFO: `/v3/projects/1bcea60d87c5481abbf565c218161a9b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:02.275178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cfb2f-7746-43c4-8ff9-8acb38faa675 None admin] RBAC: Authorizing `identity:get_project(project_id=1bcea60d87c5481abbf565c218161a9b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:02.276637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cfb2f-7746-43c4-8ff9-8acb38faa675 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:02.278211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 425/855] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:23:02 2022] GET /identity/v3/projects/1bcea60d87c5481abbf565c218161a9b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:02.284235 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:02.355525 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:02.379564 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711c718-815e-4dfc-9b60-1cf2e4b9de00 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:02.412291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:02.412888 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:02.413167 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:02.413435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:02.414144 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] RBAC: Adding query filter params (role.id=517b2b1aaac34e92b002aec2385431d0, scope.project.id=1bcea60d87c5481abbf565c218161a9b, user.id=e11c7496e35447a1aea9774c2980130e) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:02.414447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:02.415902 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce42da1-7187-4e3d-98c9-cdc09383db84 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:02.421724 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 431/856] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:23:02 2022] GET /identity/v3/role_assignments?user.id=e11c7496e35447a1aea9774c2980130e&scope.project.id=1bcea60d87c5481abbf565c218161a9b&role.id=517b2b1aaac34e92b002aec2385431d0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:04.516631 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372899c3-403c-40cb-8776-31c4be3d1d6e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:04.517175 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372899c3-403c-40cb-8776-31c4be3d1d6e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:04.517175 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372899c3-403c-40cb-8776-31c4be3d1d6e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:04.518983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 426/857] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:04.524107 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:04.524509 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:04.524657 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:04.543009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:04.544210 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:04.607003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 432/858] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:04.871787 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372899c3-403c-40cb-8776-31c4be3d1d6e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:04.875531 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:04.969649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372899c3-403c-40cb-8776-31c4be3d1d6e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:05.001060 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108d819-d9ec-4533-803c-fdbb49f9a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:05.001670 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108d819-d9ec-4533-803c-fdbb49f9a80a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:05.001948 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108d819-d9ec-4533-803c-fdbb49f9a80a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:05.002219 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108d819-d9ec-4533-803c-fdbb49f9a80a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:05.003860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108d819-d9ec-4533-803c-fdbb49f9a80a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:05.005258 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108d819-d9ec-4533-803c-fdbb49f9a80a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:05.006788 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 427/859] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:23:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:05.012529 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:05.076091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:05.108136 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a586b-3ca9-4f83-943b-8ebc27f12722 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:05.140553 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010bfda-2176-464b-8812-bd5afa34cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:05.141089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010bfda-2176-464b-8812-bd5afa34cb49 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:05.141367 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010bfda-2176-464b-8812-bd5afa34cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:05.141639 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010bfda-2176-464b-8812-bd5afa34cb49 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:05.142277 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010bfda-2176-464b-8812-bd5afa34cb49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:05.143770 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010bfda-2176-464b-8812-bd5afa34cb49 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:05.272661 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 433/860] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:23:05 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:23:07.425583 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76bb4b-b7e3-4c36-a416-37678914b2a7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:07.426075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76bb4b-b7e3-4c36-a416-37678914b2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:07.426171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76bb4b-b7e3-4c36-a416-37678914b2a7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:07.427509 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 428/861] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:07.432743 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:07.433176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:07.433282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:07.450620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:07.451816 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:07.514425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 434/862] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:07.782253 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76bb4b-b7e3-4c36-a416-37678914b2a7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:07.850809 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:07.879180 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76bb4b-b7e3-4c36-a416-37678914b2a7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:07.913220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80597efe-c2a5-4fda-8b21-4bb4809b7195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:07.913822 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597efe-c2a5-4fda-8b21-4bb4809b7195 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:07.914097 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597efe-c2a5-4fda-8b21-4bb4809b7195 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:07.914362 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597efe-c2a5-4fda-8b21-4bb4809b7195 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:07.915460 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80597efe-c2a5-4fda-8b21-4bb4809b7195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:07.916860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80597efe-c2a5-4fda-8b21-4bb4809b7195 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:07.918363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 429/863] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:07.923980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:07.971980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:08.019526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a7c55-5463-4a70-9e68-9f7fb22ae3ab None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:08.052653 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9395fcd-af18-4fc2-97a5-29e917a59ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:08.053185 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9395fcd-af18-4fc2-97a5-29e917a59ab2 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:08.053460 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9395fcd-af18-4fc2-97a5-29e917a59ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:08.053727 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9395fcd-af18-4fc2-97a5-29e917a59ab2 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:08.054368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9395fcd-af18-4fc2-97a5-29e917a59ab2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:08.055804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9395fcd-af18-4fc2-97a5-29e917a59ab2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:08.134476 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 435/864] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:23:07 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:23:10.262291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb1e1b-f84e-4920-88df-385e594565f5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:10.262852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb1e1b-f84e-4920-88df-385e594565f5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:10.262852 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb1e1b-f84e-4920-88df-385e594565f5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:10.264204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 430/865] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:10.269851 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:10.270244 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:10.270396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:10.288932 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:10.290106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:10.352559 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 436/866] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:10.617065 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb1e1b-f84e-4920-88df-385e594565f5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:10.714079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb1e1b-f84e-4920-88df-385e594565f5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:10.746855 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:10.754073 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:10.754073 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:10.754073 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:10.755338 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:10.758220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:23:10.762959 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:23:10.764183 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:23:10.765197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 431/867] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:23:10.770079 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:10.803941 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:10.866127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ba88a-8b66-45c1-90d6-84d3eb7e1797 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:10.899368 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:10.899933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:10.900218 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:10.900482 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:10.901131 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:10.901429 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:10.902784 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:10.910543 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 437/868] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:10.916624 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:11.012410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d855ba8-661b-4ac1-829f-f21f70977dc7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:11.045182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:11.045800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:11.046080 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:11.046352 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] PATH_INFO: `/v3/users/2f5dcb482bdf4c24aefb27fd4bb78e88` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:11.058759 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] RBAC: Authorizing `identity:get_user(user_id=2f5dcb482bdf4c24aefb27fd4bb78e88)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:11.060204 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:11.061765 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 432/869] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:23:10 2022] GET /identity/v3/users/2f5dcb482bdf4c24aefb27fd4bb78e88 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:11.068118 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:11.161924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:11.165509 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce1ac-c23d-4999-8365-ba4fb741056b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:11.197926 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a9c0f-9246-4e56-8037-8913543d38ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:11.198532 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a9c0f-9246-4e56-8037-8913543d38ec None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:11.198811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a9c0f-9246-4e56-8037-8913543d38ec None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:11.199080 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a9c0f-9246-4e56-8037-8913543d38ec None admin] PATH_INFO: `/v3/projects/e4c7247e811f406082e15ad1e5b8af87` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:11.200183 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a9c0f-9246-4e56-8037-8913543d38ec None admin] RBAC: Authorizing `identity:get_project(project_id=e4c7247e811f406082e15ad1e5b8af87)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:11.201601 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a9c0f-9246-4e56-8037-8913543d38ec None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:11.203176 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 438/870] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:23:11 2022] GET /identity/v3/projects/e4c7247e811f406082e15ad1e5b8af87 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:11.209457 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:11.264986 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:11.305612 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730a18b-0de6-44af-a34c-0aac6cb96137 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:11.338247 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228e3c6-ed79-4a5b-9611-fcdc346998af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:11.338882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228e3c6-ed79-4a5b-9611-fcdc346998af None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:11.339161 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228e3c6-ed79-4a5b-9611-fcdc346998af None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:11.339469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228e3c6-ed79-4a5b-9611-fcdc346998af None admin] PATH_INFO: `/v3/projects/e4c7247e811f406082e15ad1e5b8af87/users/2f5dcb482bdf4c24aefb27fd4bb78e88/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:11.341334 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228e3c6-ed79-4a5b-9611-fcdc346998af None admin] RBAC: Authorizing `identity:create_grant(project_id=e4c7247e811f406082e15ad1e5b8af87, user_id=2f5dcb482bdf4c24aefb27fd4bb78e88, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:11.342730 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228e3c6-ed79-4a5b-9611-fcdc346998af None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:11.432337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 433/871] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:23:11 2022] PUT /identity/v3/projects/e4c7247e811f406082e15ad1e5b8af87/users/2f5dcb482bdf4c24aefb27fd4bb78e88/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:23:13.530318 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca2fa-da80-4c31-8d85-2c9272a9b7c2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:13.530835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca2fa-da80-4c31-8d85-2c9272a9b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:13.530835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca2fa-da80-4c31-8d85-2c9272a9b7c2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:13.532213 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 439/872] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:13.537805 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:13.538206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:13.538359 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:13.556461 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:13.557634 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:13.621412 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 434/873] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:13.886774 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca2fa-da80-4c31-8d85-2c9272a9b7c2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:13.983775 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca2fa-da80-4c31-8d85-2c9272a9b7c2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:14.016049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:14.016654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:14.016933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:14.017204 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:14.026889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:23:14.031602 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:23:14.032682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:23:14.033533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 440/874] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:23:14.039495 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:14.085636 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:14.133499 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f0d92-68a3-4b27-921d-c65a55e5f6f6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:14.166992 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:14.167692 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:14.167974 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:14.168244 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:14.168891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:14.169191 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:14.170530 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:14.178150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 435/875] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:14.183940 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:14.210952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:14.279444 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1928c-d22c-46c8-b6ed-b54d7cc71ab8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:14.311924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:14.312528 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:14.312806 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:14.313079 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] PATH_INFO: `/v3/users/2f5dcb482bdf4c24aefb27fd4bb78e88` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:14.314165 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] RBAC: Authorizing `identity:get_user(user_id=2f5dcb482bdf4c24aefb27fd4bb78e88)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:14.315640 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:14.317163 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 441/876] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:23:14 2022] GET /identity/v3/users/2f5dcb482bdf4c24aefb27fd4bb78e88 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:14.322709 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:14.418422 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a750e-1626-4921-9958-ab7d23e31f9a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:14.428346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:14.450844 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:14.451458 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:14.451778 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:14.452045 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:14.517234 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:14.519236 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:14.611382 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 436/877] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:23:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:23:14.617261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:14.708032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:14.714350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae13b3-3798-49bb-a93f-6eff9debfe98 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:14.746152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59526ba3-d517-4cea-af10-a96b72b9de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:14.746886 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59526ba3-d517-4cea-af10-a96b72b9de52 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:14.747224 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59526ba3-d517-4cea-af10-a96b72b9de52 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:14.747969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59526ba3-d517-4cea-af10-a96b72b9de52 None admin] PATH_INFO: `/v3/projects/e4c7247e811f406082e15ad1e5b8af87` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:14.749051 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59526ba3-d517-4cea-af10-a96b72b9de52 None admin] RBAC: Authorizing `identity:get_project(project_id=e4c7247e811f406082e15ad1e5b8af87)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:14.750468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59526ba3-d517-4cea-af10-a96b72b9de52 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:14.752091 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 442/878] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:23:14 2022] GET /identity/v3/projects/e4c7247e811f406082e15ad1e5b8af87 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:14.758537 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:14.770224 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:14.853324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e83ff5-063f-4d1f-a4bf-5321374b4916 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:14.885741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:14.886327 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:14.886605 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:14.886878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:14.887619 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=e4c7247e811f406082e15ad1e5b8af87, user.id=2f5dcb482bdf4c24aefb27fd4bb78e88) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:14.887928 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:14.889317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719bb8c-e525-42f5-b5a3-563008667d87 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:14.895282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 437/879] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:23:14 2022] GET /identity/v3/role_assignments?user.id=2f5dcb482bdf4c24aefb27fd4bb78e88&scope.project.id=e4c7247e811f406082e15ad1e5b8af87&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:17.014358 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5275f5-950d-45d6-943c-2ac9ae02178e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:17.014920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5275f5-950d-45d6-943c-2ac9ae02178e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:17.014920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5275f5-950d-45d6-943c-2ac9ae02178e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:17.016247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 443/880] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:17.021565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3eb6a1-d057-4822-9374-8355a0ce2e3f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:17.021934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3eb6a1-d057-4822-9374-8355a0ce2e3f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:17.022112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3eb6a1-d057-4822-9374-8355a0ce2e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:17.040504 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9b3eb6a1-d057-4822-9374-8355a0ce2e3f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:17.041676 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9b3eb6a1-d057-4822-9374-8355a0ce2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:17.105898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 438/881] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:17.373330 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5275f5-950d-45d6-943c-2ac9ae02178e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:17.480450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5275f5-950d-45d6-943c-2ac9ae02178e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:17.513312 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d2be-c2f0-481f-b461-99af9d6a2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:17.513904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d2be-c2f0-481f-b461-99af9d6a2b22 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:17.514180 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d2be-c2f0-481f-b461-99af9d6a2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:17.514451 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d2be-c2f0-481f-b461-99af9d6a2b22 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:17.515108 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9d2be-c2f0-481f-b461-99af9d6a2b22 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:17.516516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9d2be-c2f0-481f-b461-99af9d6a2b22 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:17.650110 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 444/882] 10.222.0.13 () {60 vars in 1138 bytes} [Mon Oct 17 05:23:17 2022] POST /identity/v3/domains => generated 267 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:23:19.793001 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec84a-23c3-45d0-9082-43886b154a51 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:19.793513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec84a-23c3-45d0-9082-43886b154a51 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:19.793513 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec84a-23c3-45d0-9082-43886b154a51 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:19.794791 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 439/883] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:19.800028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:19.800468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:19.800561 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:19.818374 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:19.819570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:19.835831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:19.883271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 445/884] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:23:20.148832 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec84a-23c3-45d0-9082-43886b154a51 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:20.152602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:20.245171 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec84a-23c3-45d0-9082-43886b154a51 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:20.277026 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4bcf-97a0-4389-a9b2-cb09e13409af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:20.277638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4bcf-97a0-4389-a9b2-cb09e13409af None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:20.277916 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4bcf-97a0-4389-a9b2-cb09e13409af None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:20.278188 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4bcf-97a0-4389-a9b2-cb09e13409af None admin] PATH_INFO: `/v3/domains/cec43e2e406a48ef82e6ca245ea4f133` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:20.289000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4bcf-97a0-4389-a9b2-cb09e13409af None admin] RBAC: Authorizing `identity:get_domain(domain_id=cec43e2e406a48ef82e6ca245ea4f133)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:20.290405 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4bcf-97a0-4389-a9b2-cb09e13409af None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:20.291961 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 440/885] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Oct 17 05:23:20 2022] GET /identity/v3/domains/cec43e2e406a48ef82e6ca245ea4f133 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:20.297784 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:20.390320 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce4c0d-2e55-4fbc-b245-3b6760541cd4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:20.423614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4665063-010d-4df6-ad6b-948fe123cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:20.424266 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4665063-010d-4df6-ad6b-948fe123cd25 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:20.424548 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4665063-010d-4df6-ad6b-948fe123cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:20.424822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4665063-010d-4df6-ad6b-948fe123cd25 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:20.425468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4665063-010d-4df6-ad6b-948fe123cd25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:20.426894 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4665063-010d-4df6-ad6b-948fe123cd25 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:20.546670 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 446/886] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:23:20 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:23:22.681785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5437b-832d-4721-9401-a1d23b9cc0f1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:22.682291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5437b-832d-4721-9401-a1d23b9cc0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:22.682291 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5437b-832d-4721-9401-a1d23b9cc0f1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:22.683602 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 441/887] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:22.688853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:22.689276 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:22.689383 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:22.697032 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:22.708075 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:22.709254 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:22.774134 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 447/888] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:23.049070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5437b-832d-4721-9401-a1d23b9cc0f1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:23.109701 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:23.146432 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5437b-832d-4721-9401-a1d23b9cc0f1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:23.178903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6bafa-695c-4ee5-8459-35ac7de88173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:23.179539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6bafa-695c-4ee5-8459-35ac7de88173 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:23.179814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6bafa-695c-4ee5-8459-35ac7de88173 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:23.180079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6bafa-695c-4ee5-8459-35ac7de88173 None admin] PATH_INFO: `/v3/domains/cec43e2e406a48ef82e6ca245ea4f133` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:23.181161 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6bafa-695c-4ee5-8459-35ac7de88173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cec43e2e406a48ef82e6ca245ea4f133)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:23.182551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6bafa-695c-4ee5-8459-35ac7de88173 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:23.184098 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 442/889] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Oct 17 05:23:23 2022] GET /identity/v3/domains/cec43e2e406a48ef82e6ca245ea4f133 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:23.189896 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:23.284442 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bd817-2d3e-42bd-91cf-59192ab6d701 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:23.317980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f873441-3982-4cf6-b7f0-7b42dc27fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:23.318520 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873441-3982-4cf6-b7f0-7b42dc27fea2 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:23.318798 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873441-3982-4cf6-b7f0-7b42dc27fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:23.319069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873441-3982-4cf6-b7f0-7b42dc27fea2 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:23.319740 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f873441-3982-4cf6-b7f0-7b42dc27fea2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:23.321145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f873441-3982-4cf6-b7f0-7b42dc27fea2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:23.328861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:23.420369 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 448/890] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:23:23 2022] POST /identity/v3/users => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:23:25.530417 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be1bc-2842-4685-bf96-079eac0f87a7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:25.530970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be1bc-2842-4685-bf96-079eac0f87a7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:25.530970 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be1bc-2842-4685-bf96-079eac0f87a7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:25.532309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 443/891] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:25.537889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:25.538271 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:25.538430 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:25.556540 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:25.557706 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:25.620968 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 449/892] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:25.889206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1be1bc-2842-4685-bf96-079eac0f87a7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:25.987660 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1be1bc-2842-4685-bf96-079eac0f87a7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:26.014837 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:26.019409 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:26.020024 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:26.020302 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:26.020574 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:26.030497 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:23:26.035798 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:23:26.037248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:23:26.038153 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 444/893] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:23:26.043152 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:26.137319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30056fab-077d-46ef-aa1c-d94e3ef452cb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:26.156615 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:26.170464 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:26.171350 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:26.171465 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:26.171648 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:26.172290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:26.172585 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:26.174010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:26.181632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 450/894] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:26.187267 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:26.282297 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcaed3-8ac2-4c29-b95f-75182e0c5b89 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:26.314211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:26.314844 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:26.315123 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:26.315423 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] PATH_INFO: `/v3/users/b2b8af14eaf34ee49120ab8e68df3f19` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:26.327777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] RBAC: Authorizing `identity:get_user(user_id=b2b8af14eaf34ee49120ab8e68df3f19)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:26.329199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:26.331279 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 445/895] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:23:26 2022] GET /identity/v3/users/b2b8af14eaf34ee49120ab8e68df3f19 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:26.336999 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:26.433065 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64739f-bef1-426d-94a7-0e027633c4a6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:26.465501 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef26fc-1c50-4039-b38e-2230aa2347c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:26.466117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef26fc-1c50-4039-b38e-2230aa2347c6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:26.466400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef26fc-1c50-4039-b38e-2230aa2347c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:26.466674 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef26fc-1c50-4039-b38e-2230aa2347c6 None admin] PATH_INFO: `/v3/projects/719d99227d0d49ef8b1eac4cb89bac70` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:26.467853 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef26fc-1c50-4039-b38e-2230aa2347c6 None admin] RBAC: Authorizing `identity:get_project(project_id=719d99227d0d49ef8b1eac4cb89bac70)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:26.469280 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef26fc-1c50-4039-b38e-2230aa2347c6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:26.470935 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 451/896] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:23:26 2022] GET /identity/v3/projects/719d99227d0d49ef8b1eac4cb89bac70 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:23:26.476732 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:26.524415 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:26.571435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dad578-4750-4825-bb50-b27eaf4f24d3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:26.603431 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f678e08-412f-40a7-87c4-f672f0f45cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:26.604065 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f678e08-412f-40a7-87c4-f672f0f45cdd None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:26.604341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f678e08-412f-40a7-87c4-f672f0f45cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:26.604614 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f678e08-412f-40a7-87c4-f672f0f45cdd None admin] PATH_INFO: `/v3/projects/719d99227d0d49ef8b1eac4cb89bac70/users/b2b8af14eaf34ee49120ab8e68df3f19/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:26.606450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f678e08-412f-40a7-87c4-f672f0f45cdd None admin] RBAC: Authorizing `identity:create_grant(project_id=719d99227d0d49ef8b1eac4cb89bac70, user_id=b2b8af14eaf34ee49120ab8e68df3f19, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:26.607878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f678e08-412f-40a7-87c4-f672f0f45cdd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:26.665652 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 446/897] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:23:26 2022] PUT /identity/v3/projects/719d99227d0d49ef8b1eac4cb89bac70/users/b2b8af14eaf34ee49120ab8e68df3f19/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:23:28.763322 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaa93c-2543-4cf7-8b99-59dbfee2cee0 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:28.763822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaa93c-2543-4cf7-8b99-59dbfee2cee0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:28.763822 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaa93c-2543-4cf7-8b99-59dbfee2cee0 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:28.765130 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 452/898] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:23:28.770250 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:28.770687 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:28.770793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:28.789135 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:23:28.790308 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:23:28.854075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 447/899] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:23:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:23:29.124447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaa93c-2543-4cf7-8b99-59dbfee2cee0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:29.174566 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:29.223533 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaa93c-2543-4cf7-8b99-59dbfee2cee0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:29.256458 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:29.257071 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:29.257348 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:29.257620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:29.268199 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:23:29.272900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:23:29.274015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:23:29.274861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 453/900] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:23:29.280218 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:29.321075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:29.376920 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87314db7-c99e-4b17-a68a-d1d6727081f9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:29.409714 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:29.410320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:29.410599 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:29.410869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:29.411886 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:29.412188 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:29.413540 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:29.421662 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 448/901] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:29.428163 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:29.522429 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47945938-7be4-4980-b74f-47f8e8fd490d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:29.531992 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:29.555435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:29.556045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:29.556345 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:29.556616 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] PATH_INFO: `/v3/users/b2b8af14eaf34ee49120ab8e68df3f19` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:29.557690 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] RBAC: Authorizing `identity:get_user(user_id=b2b8af14eaf34ee49120ab8e68df3f19)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:29.559105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:29.560676 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 454/902] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:23:29 2022] GET /identity/v3/users/b2b8af14eaf34ee49120ab8e68df3f19 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:29.566336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:29.662627 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb44b29-0553-4e63-9c42-ffa18f2f3238 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:29.664069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:29.695201 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:29.695825 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:29.696102 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:29.696372 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:29.759809 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:29.761214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:29.854295 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 449/903] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:23:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:23:29.860057 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:29.956802 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9639f81-93f8-45ea-8a7f-e30a7030edb2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:29.988412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcde20-5d75-414f-9d6e-707509d83546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:29.989013 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcde20-5d75-414f-9d6e-707509d83546 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:29.989291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcde20-5d75-414f-9d6e-707509d83546 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:29.989570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcde20-5d75-414f-9d6e-707509d83546 None admin] PATH_INFO: `/v3/projects/719d99227d0d49ef8b1eac4cb89bac70` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:29.990644 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcde20-5d75-414f-9d6e-707509d83546 None admin] RBAC: Authorizing `identity:get_project(project_id=719d99227d0d49ef8b1eac4cb89bac70)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:29.992093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcde20-5d75-414f-9d6e-707509d83546 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:29.993672 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 455/904] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:23:29 2022] GET /identity/v3/projects/719d99227d0d49ef8b1eac4cb89bac70 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:29.999638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:23:30.001739 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:30.095176 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e5a0-57d1-4416-accd-ab0607f79f51 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:23:30.128340 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:23:30.128925 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:23:30.129202 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:23:30.129472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:23:30.130182 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=719d99227d0d49ef8b1eac4cb89bac70, user.id=b2b8af14eaf34ee49120ab8e68df3f19) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:23:30.130490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:23:30.131923 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec027cb-3fb2-475e-b596-2ab0aa94cbc8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:23:30.137479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 450/905] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:23:29 2022] GET /identity/v3/role_assignments?user.id=b2b8af14eaf34ee49120ab8e68df3f19&scope.project.id=719d99227d0d49ef8b1eac4cb89bac70&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:23:39.983937 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=77 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:23:40.134952 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=67 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:39.355849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623c1c-4ef4-4011-88d8-f86c64386229 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:39.356445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623c1c-4ef4-4011-88d8-f86c64386229 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:39.356445 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623c1c-4ef4-4011-88d8-f86c64386229 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:39.357716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 456/906] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:24:39.363657 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:39.364077 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:39.364248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:39.382107 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:24:39.383290 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:24:39.449837 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 451/907] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:24:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:24:39.723040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12623c1c-4ef4-4011-88d8-f86c64386229 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:39.823860 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12623c1c-4ef4-4011-88d8-f86c64386229 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:39.855899 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:39.856512 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:39.856788 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:39.857049 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:39.867124 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:39.871556 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:39.883191 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:39.884400 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:39.885621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:39.885621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:39.885621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:39.885621 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 457/908] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:39.890533 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:39.986435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6467e88-0b00-423d-9c16-8ecc697d20bc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:40.016433 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:40.022293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:40.026438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:40.026438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:40.026438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:40.026438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:40.026438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:40.026438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd81df-7b77-4af7-a3fb-aae7d052e6fb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:40.037055 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 452/909] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:40.043000 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:40.094531 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:40.138679 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782e281-180b-4c3d-ac2a-bfe7ab31683c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:40.171027 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791803-d0cc-4a15-9868-d141b5d2f267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:40.172206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791803-d0cc-4a15-9868-d141b5d2f267 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:40.172479 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791803-d0cc-4a15-9868-d141b5d2f267 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:40.172744 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791803-d0cc-4a15-9868-d141b5d2f267 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:40.173374 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a791803-d0cc-4a15-9868-d141b5d2f267 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:40.174777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a791803-d0cc-4a15-9868-d141b5d2f267 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:40.264473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 458/910] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:24:40 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:24:42.412704 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67242079-8abb-472f-8f3d-8f18d0b49ce6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:42.413102 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67242079-8abb-472f-8f3d-8f18d0b49ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:42.413252 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67242079-8abb-472f-8f3d-8f18d0b49ce6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:42.414578 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 453/911] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:24:42.419846 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:42.424290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:42.424290 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:42.438854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:24:42.440047 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:24:42.504655 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 459/912] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:24:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:24:42.774944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67242079-8abb-472f-8f3d-8f18d0b49ce6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:42.873192 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67242079-8abb-472f-8f3d-8f18d0b49ce6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:42.904910 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:42.906243 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:42.906516 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:42.906781 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:42.917404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:24:42.922164 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:42.923320 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:42.924248 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 454/913] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:24:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:42.929254 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:42.968003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:43.025483 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b17b0-d7c0-4629-92a4-d6229b68bb30 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:43.059288 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:43.059914 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:43.060354 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:43.060668 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:43.061831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:43.062127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:43.063568 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:43.071285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 460/914] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:43.076964 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:43.125723 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:43.174982 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc400f63-b985-4aa5-a20d-15d6a1d32582 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:43.207827 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:43.208446 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:43.208726 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:43.209000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:43.219695 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:43.221112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:43.235332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:43.236420 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:43.237323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:43.237323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:43.237323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:43.237323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 455/915] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:43.242272 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:43.336119 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:43.338944 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a082ee-9c5c-4d1a-99b9-7a9ad5b980b4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:43.370698 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:43.371278 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:43.371578 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:43.371842 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:43.372456 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:43.372800 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:43.374177 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:43.384716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 461/916] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:43.390864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:43.489054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc508d9-5dc4-43d3-b76a-72e7020b63b6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:43.490532 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:43.520934 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:43.521540 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:43.521816 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:43.522082 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:43.529175 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:43.538252 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:24:43.542112 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:43.543103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:43.544366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 456/917] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:43.549270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:43.645806 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b71e36-bff8-4a86-8df9-61a4c2a635c3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:43.679081 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:43.679713 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:43.679992 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:43.680261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:43.680970 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:43.681308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:43.682726 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:43.699889 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 462/918] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:24:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:43.705860 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:43.809512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95e66d-156c-40f6-b413-012ea15cf3f0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:43.841150 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:43.841762 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:43.842033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:43.842297 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:43.853855 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:43.855260 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:43.869398 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:43.870514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:43.871794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:43.871794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:43.871794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:43.871794 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 457/919] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:43.876361 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:43.894833 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:43.975114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c1c1a-3936-47f8-a537-bf59282406b9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:44.007716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:44.008308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:44.008586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:44.008854 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:44.009471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:44.009767 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:44.011124 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:44.021839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 463/920] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:44.027539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:44.044070 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:44.124336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dd24e-4c6f-41a4-b0b3-5c474eb325a8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:44.156486 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:44.157093 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:44.157368 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:44.157637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:44.164239 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:44.165647 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:24:44.177003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:44.178101 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:44.179418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 458/921] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:44.183891 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:44.260068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:44.280282 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c9698-7d72-4ed2-8873-525f51080c52 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:44.312996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:44.313587 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:44.313867 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:44.314139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:44.314807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:44.315120 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:44.317008 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2f4d1-25b9-4473-bc95-1fb275de3ee0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:44.328805 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 464/922] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:44.334633 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:44.408888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:44.432745 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e035e89-d624-4df3-90da-ce52b3125ec0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:44.466435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073f0c6-9c17-430b-973a-c6df655c8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:44.467079 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f0c6-9c17-430b-973a-c6df655c8da3 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:44.467389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f0c6-9c17-430b-973a-c6df655c8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:44.467659 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f0c6-9c17-430b-973a-c6df655c8da3 None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/45ab5dca714b4b8abfa8b2226f58de43/roles/3c55026dc6a443b18154e4e1e704d9f9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:44.481305 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073f0c6-9c17-430b-973a-c6df655c8da3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=45ab5dca714b4b8abfa8b2226f58de43, role_id=3c55026dc6a443b18154e4e1e704d9f9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:44.482791 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073f0c6-9c17-430b-973a-c6df655c8da3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:44.573211 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 459/923] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:24:44 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/45ab5dca714b4b8abfa8b2226f58de43/roles/3c55026dc6a443b18154e4e1e704d9f9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:24:46.697819 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e513dd-da0c-487a-8a98-3176beeee614 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:46.698322 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e513dd-da0c-487a-8a98-3176beeee614 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:46.698322 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e513dd-da0c-487a-8a98-3176beeee614 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:46.699637 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 465/924] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:24:46.704815 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:46.705209 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:46.706251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:46.723317 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:24:46.724512 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:24:46.789115 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 460/925] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:24:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:24:47.062373 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e513dd-da0c-487a-8a98-3176beeee614 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:47.160379 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e513dd-da0c-487a-8a98-3176beeee614 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:47.192839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:47.193435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:47.193708 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:47.193972 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:47.200486 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:47.204693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:24:47.209337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:47.213620 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:47.214659 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 466/926] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:47.216598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:47.270274 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:47.313951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293b8ff-1753-45dc-afda-d784688c38be None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:47.345514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:47.346103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:47.346377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:47.346645 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:47.347284 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:47.347611 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:47.348947 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:47.356518 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 461/927] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:47.362926 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:47.463278 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ee60-ebad-4bf3-899a-b4bfa28e3f7e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:47.496671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:47.497296 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:47.497576 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:47.497925 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:47.508229 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:47.509765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:47.524471 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:47.529776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:47.531395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:47.531395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:47.531395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:47.531395 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 467/928] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:47.531933 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:47.628357 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403b78d-bc3b-4707-8a3c-db5f2e0539b3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:47.644336 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:47.660981 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:47.661572 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:47.661846 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:47.662111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:47.662995 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:47.663289 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:47.664637 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:47.675790 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 462/929] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:47.681470 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:47.730906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:47.777873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c337a-f00c-4e50-a5c4-3f545aae4355 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:47.811675 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:47.812314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:47.812598 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:47.812869 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:47.820201 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:47.821623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:24:47.833009 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:47.834222 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:47.835473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 468/930] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:47.839953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:47.935363 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddae544-594c-4861-b7d8-147a22d030c8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:47.967476 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:47.969038 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:47.969038 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:47.969256 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:47.969963 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:47.970265 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:47.971696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:47.988767 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 463/931] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:47.994750 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:48.091387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925dd2-5747-436b-9ea6-9915cedcbed8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:48.101683 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:48.123726 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:48.124334 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:48.124609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:48.124874 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:48.189573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:48.190938 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:48.284284 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 469/932] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:24:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:24:48.290479 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:48.323585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:48.386618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5969ebf-25fe-492b-9ddb-53dbc392e080 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:48.420159 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:48.420779 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:48.421054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:48.421323 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:48.431045 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:48.432490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:48.446969 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:48.448137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:48.449324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:48.449324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:48.449324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:48.449324 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 464/933] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:48.454012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:48.465236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:48.549751 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94492e39-8671-4956-85f6-18664ac7953b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:48.582924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:48.583546 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:48.584979 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:48.584979 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:48.584979 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:48.585216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:48.586351 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:48.597286 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 470/934] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:24:48.603012 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:48.685453 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:48.700208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780490-d0e3-4616-be29-323d83a7be16 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:48.731966 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:48.735527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:48.735527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:48.735527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:48.739958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:48.741373 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:24:48.752618 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:48.753643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:48.754717 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 465/935] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:48.759804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:48.826319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:48.855973 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1e8be-7cda-4dcc-8c77-3a3c98a52d86 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:48.887396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:48.887983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:48.888258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:48.888526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:48.889188 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:48.890006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:48.891448 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739eea-1033-4bea-9066-12fcc6644a2b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:48.903206 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 471/936] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:48.910085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:49.006505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66015e7-e83e-4434-9cff-29e2a282b3b0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:49.039156 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:49.039770 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:49.040049 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:49.040322 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:49.041060 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] RBAC: Adding query filter params (role.id=3c55026dc6a443b18154e4e1e704d9f9, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=45ab5dca714b4b8abfa8b2226f58de43) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:49.041360 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:49.042753 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d96976-9636-4284-8b47-21f36cc573a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:49.048285 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 466/937] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:24:48 2022] GET /identity/v3/role_assignments?user.id=45ab5dca714b4b8abfa8b2226f58de43&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=3c55026dc6a443b18154e4e1e704d9f9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:51.173897 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f656c-cb61-48b2-9002-0374615d7ee2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:51.174435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f656c-cb61-48b2-9002-0374615d7ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:51.174435 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f656c-cb61-48b2-9002-0374615d7ee2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:51.175811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 472/938] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:24:51.180875 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:51.181294 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:51.181421 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:51.199562 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:24:51.200756 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:24:51.204018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:51.265500 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 467/939] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:24:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:24:51.531703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f656c-cb61-48b2-9002-0374615d7ee2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:51.631028 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f656c-cb61-48b2-9002-0374615d7ee2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:51.658104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:51.663839 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:51.664455 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:51.664734 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:51.665006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:51.675174 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:24:51.679975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:51.681040 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:51.682098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 473/940] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:51.689705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:51.784123 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccce2c-0948-4888-bed4-cfff87cbd6e1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:51.817456 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:51.818064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:51.818389 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:51.818665 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:51.819337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:51.819643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:51.820983 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:51.828719 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 468/941] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:24:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:51.834537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:51.929776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee004b5-c6cd-4a8b-8188-5c3904eda717 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:51.962369 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:51.962988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:51.963264 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:51.963569 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:51.973976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:51.975441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:51.989969 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:51.996123 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:51.997045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:51.997045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:51.997045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:51.997045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 474/942] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:51.997308 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:51.999920 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:52.093166 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e75842-6272-46c1-bcb3-f27a63258883 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:52.124814 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:52.125400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:52.125675 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:52.125944 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:52.126572 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:52.126868 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:52.128223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:52.139206 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 469/943] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:52.144976 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:52.191095 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:52.245542 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173f2e-4b9a-4b91-8d5f-9e3d3be4f1d8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:52.277868 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190d59-9218-489c-aa52-80f94060002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:52.278481 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190d59-9218-489c-aa52-80f94060002d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:52.278758 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190d59-9218-489c-aa52-80f94060002d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:52.279029 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190d59-9218-489c-aa52-80f94060002d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:52.286664 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56190d59-9218-489c-aa52-80f94060002d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:52.288427 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56190d59-9218-489c-aa52-80f94060002d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-56190d59-9218-489c-aa52-80f94060002d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:24:52.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:52.300387 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:52.301760 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 475/944] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:52.306196 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:52.366387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:52.402384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f364e-3bf0-487e-85d3-aff99b827de1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:52.435862 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:52.437054 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:52.437332 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:52.437604 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:52.438309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:52.438610 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:52.440033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:52.456179 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 470/945] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:24:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:52.461919 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190d59-9218-489c-aa52-80f94060002d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:52.560214 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190d59-9218-489c-aa52-80f94060002d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:52.561797 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:52.593638 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:52.594255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:52.594537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:52.594804 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:52.604317 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:52.605733 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:52.620625 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:52.621823 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:52.622745 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:52.622745 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:52.622745 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:52.622745 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 476/946] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:52.627437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:52.725299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aed178-b9dc-4bb4-9d46-ffe3becad945 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:52.744705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:52.757888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:52.758487 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:52.758764 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:52.759033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:52.759697 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:52.759997 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:52.761395 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:52.772170 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 471/947] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:52.777946 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:52.875616 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccaa1b-bacd-4d03-ae22-adf9af59ea9e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:52.908169 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:52.908779 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:52.909058 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:52.909324 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:52.915943 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:52.917443 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:24:52.928412 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:52.929570 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:52.930817 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 477/948] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:52.935503 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:53.031384 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c30bd-a700-43b9-8ec4-74fa64c35ca6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:53.063373 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:53.063965 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:53.064242 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:53.064508 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:53.065178 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:53.065478 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:53.066871 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3ab67-7295-4243-82b4-4b182a29844c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:53.079489 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 472/949] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:53.085505 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:53.101609 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:53.181107 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8754ae-818f-4076-b961-a2b6894d5665 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:53.215857 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490aafe-3dc9-4b32-bc05-3971540e565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:53.216522 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490aafe-3dc9-4b32-bc05-3971540e565c None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:53.216798 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490aafe-3dc9-4b32-bc05-3971540e565c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:53.217066 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490aafe-3dc9-4b32-bc05-3971540e565c None admin] PATH_INFO: `/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/45ab5dca714b4b8abfa8b2226f58de43/roles/16f30ecbc9e946bdb5cca05379bc9311` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:53.219526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490aafe-3dc9-4b32-bc05-3971540e565c None admin] RBAC: Authorizing `identity:create_grant(project_id=6805b8cfa1834ce7a35ab033d20909b6, user_id=45ab5dca714b4b8abfa8b2226f58de43, role_id=16f30ecbc9e946bdb5cca05379bc9311)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:53.221189 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490aafe-3dc9-4b32-bc05-3971540e565c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:53.276697 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 478/950] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:24:53 2022] PUT /identity/v3/projects/6805b8cfa1834ce7a35ab033d20909b6/users/45ab5dca714b4b8abfa8b2226f58de43/roles/16f30ecbc9e946bdb5cca05379bc9311 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:24:55.413411 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f409e-b872-491e-8791-6b2a5908a386 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:55.413894 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f409e-b872-491e-8791-6b2a5908a386 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:55.414002 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f409e-b872-491e-8791-6b2a5908a386 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:55.415318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 473/951] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:24:55.421006 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:55.423275 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:55.423275 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:55.439922 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:24:55.441209 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:24:55.506003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 479/952] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:24:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:24:55.776643 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f409e-b872-491e-8791-6b2a5908a386 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:55.819469 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:55.874667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f409e-b872-491e-8791-6b2a5908a386 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:55.908140 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:55.908763 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:55.909043 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:55.909345 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:55.919277 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:24:55.924698 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:55.926039 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:55.927177 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 474/953] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:55.931852 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:55.934904 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:56.027532 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ef42f-122d-433d-9b03-69cb64f7229c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:56.060850 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:56.061487 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:56.061764 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:56.062033 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:56.062747 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:56.063045 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:56.064487 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:56.073165 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 480/954] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:56.078818 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:56.172497 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:56.175528 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bca90-98fc-4c2a-b5e8-4c551f7f1e0a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:56.207638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:56.208243 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:56.208517 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:56.208782 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:56.218793 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:56.220229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:56.234052 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:56.235259 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:56.239598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:56.239598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:56.239598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:56.239598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 475/955] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:56.241089 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:56.304873 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:56.337764 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9fa90-aceb-4cd1-8dee-58cdefa4a274 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:56.369397 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:56.369986 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:56.370265 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:56.370535 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:56.371156 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:56.371495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:56.372909 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:56.383980 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 481/956] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:56.390438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:56.489396 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a04f2-f710-4090-9b35-5663b83c5a73 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:56.520679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:56.521293 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:56.521569 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:56.521834 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:56.528878 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:56.530795 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:24:56.541747 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:56.543833 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:24:56.545032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 476/957] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:56.549469 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:56.644893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4189-9cba-4b40-9770-46e5d4159b6b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:56.670189 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:56.679127 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:56.679754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:56.680053 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:56.680344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:56.681052 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:56.681383 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:56.682796 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:56.698956 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 482/958] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:24:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:56.704789 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:56.720069 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:56.800452 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b8dac-cf2c-45ea-8864-9fb9e5219199 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:56.832846 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:56.833437 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:56.833715 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:56.833984 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:56.899274 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:56.900658 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:56.914003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:56.995842 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 477/959] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:24:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:24:57.002230 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:57.097959 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ecc21-82d2-4510-9212-4ca785bca30f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:57.130985 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:57.131623 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:57.131901 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:57.132219 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:57.142121 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:57.143645 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:24:57.157632 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:57.158757 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:24:57.159906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:24:57.159906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:24:57.159906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:57.159906 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 483/960] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:57.164808 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:57.260560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8997-9171-40fb-a605-621c4e061aac None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:57.274085 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:57.294539 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:57.295162 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:57.295508 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:57.295785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:57.296417 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:57.296719 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:57.298121 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:57.308971 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 478/961] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:57.315095 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:57.349652 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:57.412876 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2d2c9-70c4-47a8-bf3d-a23a16ce4278 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:57.446098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:57.446708 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:57.446983 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:57.447253 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:57.453622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:57.455036 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:24:57.465900 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:24:57.466936 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 05:24:57.468835 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 484/962] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:24:57.473527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:57.570098 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bf953-12c8-4447-8d6f-8460251e9dfe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:57.603505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:57.604095 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:57.604370 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:57.604640 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:57.605339 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:57.605641 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:57.607031 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee80d8f-cf04-48b1-974e-787fee1adb9f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:57.618485 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 479/963] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:57.624534 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:24:57.714881 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:24:57.720508 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9d45d-2b6c-4091-9ca2-05733055b4b4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:24:57.753360 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:24:57.753975 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:57.754255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:57.754526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:57.755238 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] RBAC: Adding query filter params (role.id=16f30ecbc9e946bdb5cca05379bc9311, scope.project.id=6805b8cfa1834ce7a35ab033d20909b6, user.id=45ab5dca714b4b8abfa8b2226f58de43) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:24:57.755585 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:24:57.757046 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf507c6-f843-4730-917b-c70b77cd49b9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:24:57.762920 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 485/964] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:24:57 2022] GET /identity/v3/role_assignments?user.id=45ab5dca714b4b8abfa8b2226f58de43&scope.project.id=6805b8cfa1834ce7a35ab033d20909b6&role.id=16f30ecbc9e946bdb5cca05379bc9311 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:24:59.908214 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda2a7-6fb9-4c4a-be3c-da93b3d4c5e2 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:59.908742 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda2a7-6fb9-4c4a-be3c-da93b3d4c5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:59.908851 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda2a7-6fb9-4c4a-be3c-da93b3d4c5e2 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:59.910134 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 480/965] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:24:59.915716 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:24:59.925608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:24:59.925608 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:24:59.934439 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:24:59.935669 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:25:00.000211 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 486/966] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:24:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:25:00.269874 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda2a7-6fb9-4c4a-be3c-da93b3d4c5e2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:00.301003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:00.369341 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda2a7-6fb9-4c4a-be3c-da93b3d4c5e2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:00.402924 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:00.403565 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:00.403840 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:00.404103 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:00.404709 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:00.406057 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:25:00.415138 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:25:00.416296 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:00.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 481/967] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:25:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:25:00.422245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:00.509630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:00.518861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd9422-57e3-4df3-9f19-49c78a07e6ad None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:00.552467 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:00.553003 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:00.553345 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:00.553622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:00.554236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:25:00.554535 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:00.555952 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c3dab-9ec2-4b6a-b029-1accdddf2c05 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:00.561139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 487/968] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:25:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:25:00.566867 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:00.658093 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:00.664369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99dc42-c542-4a99-9173-52043b794261 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:00.697165 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:00.697777 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:00.698066 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:00.698333 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:00.698958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:25:00.699264 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:00.700659 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b6d68-21bc-4faf-9895-f088688440dd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:00.705444 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 482/969] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:25:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:25:02.825593 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d823590-32d0-4028-93f0-700595ff819e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:02.826106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d823590-32d0-4028-93f0-700595ff819e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:02.826106 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d823590-32d0-4028-93f0-700595ff819e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:02.828079 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 488/970] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:25:02.833318 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8452d-9fab-404d-9b8b-b2e4c68fed6c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:02.833705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8452d-9fab-404d-9b8b-b2e4c68fed6c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:02.833869 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8452d-9fab-404d-9b8b-b2e4c68fed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:02.851758 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-80e8452d-9fab-404d-9b8b-b2e4c68fed6c None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:25:02.852939 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-80e8452d-9fab-404d-9b8b-b2e4c68fed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:25:02.917478 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 483/971] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:25:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:25:03.184406 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d823590-32d0-4028-93f0-700595ff819e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:03.281787 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d823590-32d0-4028-93f0-700595ff819e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:03.316055 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830690f-55ad-4c7e-979f-510b64556b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:03.316646 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830690f-55ad-4c7e-979f-510b64556b4b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:03.316924 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830690f-55ad-4c7e-979f-510b64556b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:03.317195 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830690f-55ad-4c7e-979f-510b64556b4b None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:03.317827 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830690f-55ad-4c7e-979f-510b64556b4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:03.319181 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830690f-55ad-4c7e-979f-510b64556b4b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:03.384921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 489/972] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:25:03 2022] POST /identity/v3/services => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:25:05.513298 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac1b5-5b7d-4443-89c2-5241a2bbfebb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:05.513828 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac1b5-5b7d-4443-89c2-5241a2bbfebb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:05.513941 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac1b5-5b7d-4443-89c2-5241a2bbfebb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:05.515207 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 484/973] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:25:05.520246 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:05.520723 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:05.520830 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:05.539147 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:25:05.540353 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:25:05.549893 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:05.611175 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 490/974] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:25:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:25:05.875366 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac1b5-5b7d-4443-89c2-5241a2bbfebb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:05.963229 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:05.973713 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac1b5-5b7d-4443-89c2-5241a2bbfebb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:06.005345 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:06.005952 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:06.006232 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:06.006502 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:06.007077 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:06.008456 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:25:06.018064 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:25:06.022749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:06.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 485/975] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:25:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:25:06.025394 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:06.120236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33030268-da6b-4e4a-ba20-6455a9ea4685 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:06.153781 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:06.154304 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:06.155113 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:06.155416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:06.163500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:25:06.163500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:06.163500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:06.163500 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 491/976] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:25:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:25:06.169000 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:06.265514 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fb8d9-976d-40cc-add5-f4dd4e2d5839 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:06.297440 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:06.298032 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:06.298309 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:06.298579 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:06.299268 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] RBAC: Adding query filter params (interface=public, service_id=7393e7decb814504be196d77e7da8294, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:25:06.299605 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:06.300953 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8597a-898e-428f-89d6-e0f6c93a2550 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:06.305792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 486/977] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:25:06 2022] GET /identity/v3/endpoints?service_id=7393e7decb814504be196d77e7da8294&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:25:06.311776 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:06.315425 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:06.407470 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f391d-e0df-4eca-a4e5-891005ce9e6e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:06.440642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16983914-4b18-4a04-936c-47388efaeeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:06.441296 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16983914-4b18-4a04-936c-47388efaeeaf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:06.441691 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16983914-4b18-4a04-936c-47388efaeeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:06.441963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16983914-4b18-4a04-936c-47388efaeeaf None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:06.442552 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16983914-4b18-4a04-936c-47388efaeeaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:06.443928 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16983914-4b18-4a04-936c-47388efaeeaf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:06.450584 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 492/978] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:25:06 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:25:08.544843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72561ad5-1068-481a-b6a7-cbbdf23a9ae4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:08.545435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72561ad5-1068-481a-b6a7-cbbdf23a9ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:08.545435 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72561ad5-1068-481a-b6a7-cbbdf23a9ae4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:08.546757 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 487/979] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:25:08.552979 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:08.553399 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:08.553529 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:08.570922 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:25:08.572117 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:25:08.630703 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:08.635962 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 493/980] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:25:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:25:08.901418 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72561ad5-1068-481a-b6a7-cbbdf23a9ae4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:08.921268 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:08.999575 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72561ad5-1068-481a-b6a7-cbbdf23a9ae4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:09.031648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:09.032272 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:09.032553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:09.032826 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:09.033399 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:09.034741 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:25:09.043560 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:25:09.045519 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 05:25:09.046648 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 488/981] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:25:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:25:09.051081 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:09.147037 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bdcb4-8fda-4922-a6f8-2dd1cd15f7e5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:09.180215 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:09.180762 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:09.181042 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:09.181314 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:09.181933 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:25:09.182236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:09.183630 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07b706-059e-4ecd-a459-ae7ad57ec296 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:09.189531 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 494/982] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:25:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:25:09.194842 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:25:09.275172 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:09.292629 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d37c5-61f5-4eb0-87c2-dcfa4f8cbef7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:25:09.324415 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53320a1a-aef6-4834-9e00-bb2ccd179df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7f0e8d5f450484e921b5c820f033ee1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:25:09.325003 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53320a1a-aef6-4834-9e00-bb2ccd179df1 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:25:09.325278 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53320a1a-aef6-4834-9e00-bb2ccd179df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:25:09.325544 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53320a1a-aef6-4834-9e00-bb2ccd179df1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:25:09.326196 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53320a1a-aef6-4834-9e00-bb2ccd179df1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:25:09.328232 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53320a1a-aef6-4834-9e00-bb2ccd179df1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:25:09.465091 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 489/983] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:25:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:25:19.185654 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=69 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:25:19.460547 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:25.559137 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bccedc-5d24-496c-a7c5-4cc6e1c0ea7b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:25.559600 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bccedc-5d24-496c-a7c5-4cc6e1c0ea7b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:25.559754 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bccedc-5d24-496c-a7c5-4cc6e1c0ea7b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:25.561586 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 495/984] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:25.567505 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46f73f-ab03-427f-af47-d84d1a41793b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:25.573299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46f73f-ab03-427f-af47-d84d1a41793b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:25.573299 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46f73f-ab03-427f-af47-d84d1a41793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:25.708887 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ea46f73f-ab03-427f-af47-d84d1a41793b None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:25.710123 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ea46f73f-ab03-427f-af47-d84d1a41793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:25.839785 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 490/985] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:26.132573 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce7793-65b6-4e47-b451-b4c162eccaae None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:26.132953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce7793-65b6-4e47-b451-b4c162eccaae None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:26.133329 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce7793-65b6-4e47-b451-b4c162eccaae None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:26.135450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 496/986] 10.222.0.13 () {54 vars in 964 bytes} [Mon Oct 17 05:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:26.141525 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d9551-1347-45e5-9a2f-845c5faaeda9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:26.141996 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d9551-1347-45e5-9a2f-845c5faaeda9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:26.142416 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d9551-1347-45e5-9a2f-845c5faaeda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:26.232554 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-773d9551-1347-45e5-9a2f-845c5faaeda9 None None] MFA Rules not processed for user `937548ffd2d040f6a707f515faa6a4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:26.233949 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-773d9551-1347-45e5-9a2f-845c5faaeda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:26.261089 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:26.332993 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 491/987] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Oct 17 05:32:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:32:26.342739 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce7793-65b6-4e47-b451-b4c162eccaae None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:26.344834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-75ce7793-65b6-4e47-b451-b4c162eccaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:26.477967 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce7793-65b6-4e47-b451-b4c162eccaae None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:26.562997 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:26.564737 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:26.565256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:26.565751 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:26.567812 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:26.630539 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:26.695258 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:26.697145 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed7770-c6c4-48b1-9b67-b2a437436860 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:26.720502 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 497/988] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:29.200410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e8986-e8b4-4eff-bf79-74e13b44c4ad None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:29.200967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e8986-e8b4-4eff-bf79-74e13b44c4ad None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:29.200967 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e8986-e8b4-4eff-bf79-74e13b44c4ad None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:29.202325 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 492/989] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:29.208007 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7feb4c5-7640-4eff-8eca-fcab0b75e671 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:29.208457 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7feb4c5-7640-4eff-8eca-fcab0b75e671 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:29.208998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7feb4c5-7640-4eff-8eca-fcab0b75e671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:29.281180 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b7feb4c5-7640-4eff-8eca-fcab0b75e671 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:29.284093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7feb4c5-7640-4eff-8eca-fcab0b75e671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:29.376827 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 498/990] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:29.658669 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e8986-e8b4-4eff-bf79-74e13b44c4ad None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:29.660217 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cc7e8986-e8b4-4eff-bf79-74e13b44c4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:29.765647 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e8986-e8b4-4eff-bf79-74e13b44c4ad None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:29.835716 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:29.838065 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:29.838662 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:29.838942 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:29.839217 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:29.840880 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:29.948357 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:29.949705 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d822df-98f6-486d-9da3-d90c03c46ac9 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:29.967227 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 493/991] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:32.664644 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54992-86f8-4bb1-ad3d-8a24fa8a7937 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:32.665173 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54992-86f8-4bb1-ad3d-8a24fa8a7937 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:32.665173 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54992-86f8-4bb1-ad3d-8a24fa8a7937 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:32.666721 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 499/992] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:32.671865 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e9d6-8593-4d8c-b21a-0e2fbc6b09b7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:32.672288 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e9d6-8593-4d8c-b21a-0e2fbc6b09b7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:32.672400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e9d6-8593-4d8c-b21a-0e2fbc6b09b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:32.737034 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-50f1e9d6-8593-4d8c-b21a-0e2fbc6b09b7 None None] MFA Rules not processed for user `3db07d1d0f574b668fc4e337a196088f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:32.738220 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-50f1e9d6-8593-4d8c-b21a-0e2fbc6b09b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:32.788853 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:32.832228 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 494/993] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:33.112996 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54992-86f8-4bb1-ad3d-8a24fa8a7937 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:33.114514 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-01c54992-86f8-4bb1-ad3d-8a24fa8a7937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:33.124994 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:33.214353 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54992-86f8-4bb1-ad3d-8a24fa8a7937 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:33.284878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:33.285468 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:33.285744 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:33.286014 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:33.287607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:33.348898 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:33.393894 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:33.395328 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf121a-ba29-46c9-a9d1-21ef0a0f4990 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:33.414411 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 500/994] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:36.384689 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda5bb-f4b6-4c85-afe7-cd5b4e879695 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:36.385376 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda5bb-f4b6-4c85-afe7-cd5b4e879695 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:36.385376 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda5bb-f4b6-4c85-afe7-cd5b4e879695 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:36.386574 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 495/995] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:36.391914 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1a691-26f1-4959-baa0-0488c41bb4f2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:36.392362 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1a691-26f1-4959-baa0-0488c41bb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:36.392473 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1a691-26f1-4959-baa0-0488c41bb4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:36.459507 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-20d1a691-26f1-4959-baa0-0488c41bb4f2 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:36.460702 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-20d1a691-26f1-4959-baa0-0488c41bb4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:36.553878 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 501/996] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:36.834899 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda5bb-f4b6-4c85-afe7-cd5b4e879695 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:36.836449 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-09fda5bb-f4b6-4c85-afe7-cd5b4e879695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:36.936387 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda5bb-f4b6-4c85-afe7-cd5b4e879695 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:37.006161 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:37.006749 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:37.007020 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:37.007282 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:37.008843 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:37.015979 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:37.113490 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:37.114888 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e157c2-a843-43e2-910e-19b3608be64f service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:37.132498 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 496/997] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:40.299291 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46427c9c-a5db-44c8-a73d-ee3775540127 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:40.299907 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46427c9c-a5db-44c8-a73d-ee3775540127 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:40.299907 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46427c9c-a5db-44c8-a73d-ee3775540127 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:40.301323 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 502/998] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:40.306438 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d64511-49b8-4307-8356-9ebf15f3cae6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:40.307372 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d64511-49b8-4307-8356-9ebf15f3cae6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:40.307668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d64511-49b8-4307-8356-9ebf15f3cae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:40.374226 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-25d64511-49b8-4307-8356-9ebf15f3cae6 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:40.375472 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-25d64511-49b8-4307-8356-9ebf15f3cae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:40.438111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:40.481044 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 497/999] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:40.778925 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f4e5-8483-426d-976d-4484982a81b6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:40.779321 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f4e5-8483-426d-976d-4484982a81b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:40.779673 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f4e5-8483-426d-976d-4484982a81b6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:40.781093 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 503/1000] 10.222.0.13 () {54 vars in 964 bytes} [Mon Oct 17 05:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:40.787094 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b70897-fc37-4df5-be53-feda0b068a40 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:40.787407 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b70897-fc37-4df5-be53-feda0b068a40 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:40.787683 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b70897-fc37-4df5-be53-feda0b068a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:40.864447 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d9b70897-fc37-4df5-be53-feda0b068a40 None None] MFA Rules not processed for user `937548ffd2d040f6a707f515faa6a4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:40.865681 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d9b70897-fc37-4df5-be53-feda0b068a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:40.939621 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 498/1001] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Oct 17 05:32:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:40.948499 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35f4e5-8483-426d-976d-4484982a81b6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:40.950087 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5e35f4e5-8483-426d-976d-4484982a81b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:40.970304 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:41.057067 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35f4e5-8483-426d-976d-4484982a81b6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:41.129537 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:41.130062 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:41.130343 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:41.130614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:41.132241 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:41.202069 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:41.239970 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:41.241441 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4647c-cb60-4bf9-a6dc-692659b9fcc9 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:41.259238 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 504/1002] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:43.706977 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e6851-ea5f-4b28-b597-a024dbd44717 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:43.707555 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e6851-ea5f-4b28-b597-a024dbd44717 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:43.707555 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e6851-ea5f-4b28-b597-a024dbd44717 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:43.708898 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 499/1003] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:43.714260 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1eae7-41e1-4db9-8d9f-455e64335111 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:43.714686 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1eae7-41e1-4db9-8d9f-455e64335111 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:43.714930 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1eae7-41e1-4db9-8d9f-455e64335111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:43.781238 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6bd1eae7-41e1-4db9-8d9f-455e64335111 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:43.782393 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6bd1eae7-41e1-4db9-8d9f-455e64335111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:43.878777 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 505/1004] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:44.161649 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e6851-ea5f-4b28-b597-a024dbd44717 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:44.163283 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ad3e6851-ea5f-4b28-b597-a024dbd44717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:44.224348 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:44.267769 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e6851-ea5f-4b28-b597-a024dbd44717 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:44.342950 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:44.343668 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:44.343951 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:44.344223 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:44.345815 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:44.456222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:44.458659 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:44.460137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260712b5-f743-4046-99fd-18575a2b97e7 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:44.478404 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 500/1005] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:46.693125 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa212d-4dc7-4dc8-a1c6-848ba0563eda None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:46.693642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa212d-4dc7-4dc8-a1c6-848ba0563eda None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:46.693642 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa212d-4dc7-4dc8-a1c6-848ba0563eda None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:46.694989 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 506/1006] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:46.700974 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72a873-ab2a-4a7d-91e7-bc54fb78807a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:46.701388 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72a873-ab2a-4a7d-91e7-bc54fb78807a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:46.701671 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72a873-ab2a-4a7d-91e7-bc54fb78807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:46.770882 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7d72a873-ab2a-4a7d-91e7-bc54fb78807a None None] MFA Rules not processed for user `3db07d1d0f574b668fc4e337a196088f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:46.772131 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7d72a873-ab2a-4a7d-91e7-bc54fb78807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:46.875719 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 501/1007] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:47.157371 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa212d-4dc7-4dc8-a1c6-848ba0563eda None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:47.158925 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b2aa212d-4dc7-4dc8-a1c6-848ba0563eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:47.191216 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:47.266067 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa212d-4dc7-4dc8-a1c6-848ba0563eda None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:47.354365 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:47.355025 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:47.355347 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:47.355590 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:47.357249 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:47.439073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:47.470129 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:47.471588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53ed0f-e1cc-4045-9646-db9c32aae3ec service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:47.490727 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 507/1008] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:50.020400 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8453-6d6e-4683-81a0-14c961b023b9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:50.020963 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8453-6d6e-4683-81a0-14c961b023b9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:50.020963 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8453-6d6e-4683-81a0-14c961b023b9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:50.022261 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 502/1009] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:50.027398 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cfbd7-5477-459d-8743-bf4bedfef41d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:50.027809 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cfbd7-5477-459d-8743-bf4bedfef41d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:50.027963 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cfbd7-5477-459d-8743-bf4bedfef41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:50.092683 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a34cfbd7-5477-459d-8743-bf4bedfef41d None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:50.093861 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a34cfbd7-5477-459d-8743-bf4bedfef41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:50.188034 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 508/1010] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:50.464038 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad8453-6d6e-4683-81a0-14c961b023b9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:50.465553 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ccad8453-6d6e-4683-81a0-14c961b023b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:50.532985 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:50.565406 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad8453-6d6e-4683-81a0-14c961b023b9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:50.636306 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:50.636903 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:50.637185 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:50.637454 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:50.639012 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:50.745270 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:50.746603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae3ba3-2f6a-4e2c-9a0b-2b0d265e0cc9 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:50.756235 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:50.764072 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 503/1011] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:52.969947 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b83365-b625-4d5b-ac23-fe7974a37d70 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:52.970538 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b83365-b625-4d5b-ac23-fe7974a37d70 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:52.970538 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b83365-b625-4d5b-ac23-fe7974a37d70 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:52.971926 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 509/1012] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:52.977084 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c8140-cf3e-4099-a64a-190c99c4929f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:52.977481 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c8140-cf3e-4099-a64a-190c99c4929f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:52.977639 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c8140-cf3e-4099-a64a-190c99c4929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:53.043068 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fb2c8140-cf3e-4099-a64a-190c99c4929f None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:53.044306 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fb2c8140-cf3e-4099-a64a-190c99c4929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:53.138450 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 504/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:53.424144 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b83365-b625-4d5b-ac23-fe7974a37d70 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:53.425673 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-45b83365-b625-4d5b-ac23-fe7974a37d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:53.466322 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:53.528139 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b83365-b625-4d5b-ac23-fe7974a37d70 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:53.599155 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:53.599778 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:53.600054 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:53.600319 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:53.601879 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:53.682179 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:53.708318 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:53.709735 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb2e7f-72f1-4850-a404-5c03e9d8aa26 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:53.727244 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 510/1014] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:32:56.729075 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83410c96-0407-459a-b9de-5efb447a296d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:56.729639 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83410c96-0407-459a-b9de-5efb447a296d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:56.729639 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83410c96-0407-459a-b9de-5efb447a296d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:56.731007 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 505/1015] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:32:56.736233 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae6c3-6226-4eff-802c-577d1376c641 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:56.736671 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae6c3-6226-4eff-802c-577d1376c641 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:56.736785 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae6c3-6226-4eff-802c-577d1376c641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:56.802187 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fa4ae6c3-6226-4eff-802c-577d1376c641 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:32:56.803396 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fa4ae6c3-6226-4eff-802c-577d1376c641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:56.896565 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 511/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:32:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:32:57.174551 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83410c96-0407-459a-b9de-5efb447a296d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:32:57.176107 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-83410c96-0407-459a-b9de-5efb447a296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:57.251033 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:32:57.277696 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83410c96-0407-459a-b9de-5efb447a296d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:32:57.349198 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:32:57.349802 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:32:57.350078 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:32:57.350346 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:32:57.351931 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:32:57.461892 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:32:57.463815 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41faa22-50d3-4b08-b573-a21e4989fccf service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:32:57.482018 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 506/1017] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:32:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:33:01.805012 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4768-5011-418c-a002-06bd5458f87f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:01.805509 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4768-5011-418c-a002-06bd5458f87f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:01.805614 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4768-5011-418c-a002-06bd5458f87f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:01.807447 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 512/1018] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:01.812667 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86def4-f4b6-4894-80cd-3881401056ae None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:01.813086 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86def4-f4b6-4894-80cd-3881401056ae None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:01.813230 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86def4-f4b6-4894-80cd-3881401056ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:01.818788 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:01.878958 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ab86def4-f4b6-4894-80cd-3881401056ae None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:01.880197 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ab86def4-f4b6-4894-80cd-3881401056ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:01.977527 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 507/1019] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:02.264587 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4768-5011-418c-a002-06bd5458f87f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:33:02.266120 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-84df4768-5011-418c-a002-06bd5458f87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:02.316209 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:02.370010 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4768-5011-418c-a002-06bd5458f87f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:33:02.448457 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:33:02.449105 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:02.449416 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:02.449688 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:02.451279 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:02.544945 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:02.560594 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:33:02.562031 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5597dff-02e4-43cc-8982-7f6b8ef79855 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:33:02.579773 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 513/1020] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:33:05.766165 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695b53-99c7-466a-8d6a-52fa8bb951a9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:05.766724 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695b53-99c7-466a-8d6a-52fa8bb951a9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:05.766724 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695b53-99c7-466a-8d6a-52fa8bb951a9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:05.768208 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 508/1021] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:05.773253 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a766bb-fadc-49d3-965e-57f8d3c7f691 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:05.773694 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a766bb-fadc-49d3-965e-57f8d3c7f691 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:05.773831 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a766bb-fadc-49d3-965e-57f8d3c7f691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:05.839497 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f7a766bb-fadc-49d3-965e-57f8d3c7f691 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:05.840685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f7a766bb-fadc-49d3-965e-57f8d3c7f691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:05.937285 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 514/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:33:06.220021 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695b53-99c7-466a-8d6a-52fa8bb951a9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:33:06.221537 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c0695b53-99c7-466a-8d6a-52fa8bb951a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:06.310890 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:06.326974 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695b53-99c7-466a-8d6a-52fa8bb951a9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:33:06.400922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:33:06.401520 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:06.401792 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:06.402061 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:06.403686 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:06.539849 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:33:06.542189 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561fafc-bdad-4ac1-9651-61bb749890d8 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:33:06.560847 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 509/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:33:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:33:10.990489 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509a63-d874-4845-ab56-bc572a75c663 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:10.991068 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509a63-d874-4845-ab56-bc572a75c663 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:10.991162 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509a63-d874-4845-ab56-bc572a75c663 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:10.992612 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 515/1024] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:10.998669 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4700-8a35-4761-adeb-c761070f07a7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:10.999199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4700-8a35-4761-adeb-c761070f07a7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:10.999199 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4700-8a35-4761-adeb-c761070f07a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:11.015470 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:11.068530 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-eacb4700-8a35-4761-adeb-c761070f07a7 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:11.069841 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eacb4700-8a35-4761-adeb-c761070f07a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:11.168337 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 510/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:33:11.448693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509a63-d874-4845-ab56-bc572a75c663 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:33:11.450261 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c6509a63-d874-4845-ab56-bc572a75c663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:11.507607 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:11.552720 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509a63-d874-4845-ab56-bc572a75c663 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:33:11.628705 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '937548ffd2d040f6a707f515faa6a4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:33:11.629308 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:11.629588 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:11.629858 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:11.631469 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:11.741693 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:11.744786 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:33:11.746226 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb73a00-e718-49f8-8989-6677a86e5460 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:33:11.763918 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 516/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:33:16.657385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe3aa1-e6ba-4b6b-89c5-36741e4ce2f9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:16.657385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe3aa1-e6ba-4b6b-89c5-36741e4ce2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:16.657385 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe3aa1-e6ba-4b6b-89c5-36741e4ce2f9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:16.659342 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 511/1027] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 05:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:16.664165 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d787ef-0d4c-4448-ae28-8e14416a4da7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:16.664605 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d787ef-0d4c-4448-ae28-8e14416a4da7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:16.665253 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d787ef-0d4c-4448-ae28-8e14416a4da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:16.800516 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-58d787ef-0d4c-4448-ae28-8e14416a4da7 None None] MFA Rules not processed for user `45ab5dca714b4b8abfa8b2226f58de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:16.801613 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-58d787ef-0d4c-4448-ae28-8e14416a4da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:16.921765 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 517/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 05:33:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:20.963598 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4efc0-dbac-460b-a9bf-542492a29ebd None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:20.964147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4efc0-dbac-460b-a9bf-542492a29ebd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:20.964147 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4efc0-dbac-460b-a9bf-542492a29ebd None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:20.965796 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 512/1029] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 05:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:20.970843 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f6ef-14c7-48f3-b5ab-fc533cf6ed10 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:20.971276 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f6ef-14c7-48f3-b5ab-fc533cf6ed10 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:20.971450 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f6ef-14c7-48f3-b5ab-fc533cf6ed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:21.031834 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:21.055275 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1ea2f6ef-14c7-48f3-b5ab-fc533cf6ed10 None None] MFA Rules not processed for user `45ab5dca714b4b8abfa8b2226f58de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:21.056552 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1ea2f6ef-14c7-48f3-b5ab-fc533cf6ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:21.166230 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 518/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 05:33:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:21.166462 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=59 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:22.546335 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c326cd3-e774-4179-8f60-fed3c1854e77 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:22.546940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c326cd3-e774-4179-8f60-fed3c1854e77 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:22.546940 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c326cd3-e774-4179-8f60-fed3c1854e77 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:22.548175 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 513/1031] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 05:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:22.553458 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2d1bd-f5fa-4854-bedf-1aeb09964718 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:22.553898 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2d1bd-f5fa-4854-bedf-1aeb09964718 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:22.554143 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2d1bd-f5fa-4854-bedf-1aeb09964718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:22.630967 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-45c2d1bd-f5fa-4854-bedf-1aeb09964718 None None] MFA Rules not processed for user `45ab5dca714b4b8abfa8b2226f58de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:22.632236 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-45c2d1bd-f5fa-4854-bedf-1aeb09964718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:22.674622 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:22.739581 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 519/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 05:33:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:26.306402 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4cc5e-afae-48c3-a190-b2ca879143b7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:26.306922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4cc5e-afae-48c3-a190-b2ca879143b7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:26.306922 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4cc5e-afae-48c3-a190-b2ca879143b7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:26.308585 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 514/1033] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:26.313685 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8fc22-552d-4c25-9fd5-9294ac91d959 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:26.314098 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8fc22-552d-4c25-9fd5-9294ac91d959 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:26.314255 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8fc22-552d-4c25-9fd5-9294ac91d959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:26.384682 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-13b8fc22-552d-4c25-9fd5-9294ac91d959 None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:26.385827 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-13b8fc22-552d-4c25-9fd5-9294ac91d959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:26.484361 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 520/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:26.790445 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c23bd1-4c26-4165-bb31-4d8bca43b723 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:26.791328 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c23bd1-4c26-4165-bb31-4d8bca43b723 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:26.791744 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c23bd1-4c26-4165-bb31-4d8bca43b723 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:26.794813 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 515/1035] 10.222.0.13 () {54 vars in 961 bytes} [Mon Oct 17 05:33:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:26.803579 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5007ce-e122-402c-840f-c97bd1012849 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:26.803877 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5007ce-e122-402c-840f-c97bd1012849 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:26.804154 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5007ce-e122-402c-840f-c97bd1012849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:26.836921 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:26.917423 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0a5007ce-e122-402c-840f-c97bd1012849 None None] MFA Rules not processed for user `231350e78b2740a4b0aef761f7baa4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:26.918807 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0a5007ce-e122-402c-840f-c97bd1012849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:27.041915 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 521/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Oct 17 05:33:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:27.049870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c23bd1-4c26-4165-bb31-4d8bca43b723 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:33:27.051377 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c7c23bd1-4c26-4165-bb31-4d8bca43b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:27.205848 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c23bd1-4c26-4165-bb31-4d8bca43b723 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:33:27.285016 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:27.325603 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '231350e78b2740a4b0aef761f7baa4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:33:27.326207 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:27.326530 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:27.326861 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:27.328629 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:27.441908 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:33:27.444800 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e195-7061-4625-ad30-29e883a7b4ee service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:33:27.465488 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 516/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:33:29.875444 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b829-d168-445f-8b03-5150c983c13c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:29.875982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b829-d168-445f-8b03-5150c983c13c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:29.875982 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b829-d168-445f-8b03-5150c983c13c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:29.877296 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 522/1038] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:29.882543 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b36611-ff98-4118-b9ec-c33428027b0a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:29.882945 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b36611-ff98-4118-b9ec-c33428027b0a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:29.883126 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b36611-ff98-4118-b9ec-c33428027b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:29.982068 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d0b36611-ff98-4118-b9ec-c33428027b0a None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:29.983408 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d0b36611-ff98-4118-b9ec-c33428027b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:30.023679 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:30.086868 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 517/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:30.376495 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2b829-d168-445f-8b03-5150c983c13c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:33:30.378166 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dbf2b829-d168-445f-8b03-5150c983c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:30.447988 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:30.533270 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2b829-d168-445f-8b03-5150c983c13c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:33:30.649855 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '231350e78b2740a4b0aef761f7baa4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:33:30.650337 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:30.650610 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:30.650877 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:30.652540 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:30.677015 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:30.746628 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:33:30.748245 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929537-460e-43ba-b5c9-66a6d6b58fd7 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:33:30.767256 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 523/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:33:33.386638 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f83c9d-1bbe-494f-8e72-7d6d35e1372b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:33.387222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f83c9d-1bbe-494f-8e72-7d6d35e1372b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:33.387222 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f83c9d-1bbe-494f-8e72-7d6d35e1372b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:33.388702 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 518/1041] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:33.393872 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006eca8-a302-4ec2-bd8f-e3b1e4ce642d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:33.394304 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006eca8-a302-4ec2-bd8f-e3b1e4ce642d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:33.394417 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006eca8-a302-4ec2-bd8f-e3b1e4ce642d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:33.464915 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c006eca8-a302-4ec2-bd8f-e3b1e4ce642d None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:33.466114 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c006eca8-a302-4ec2-bd8f-e3b1e4ce642d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:33.565877 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:33.575344 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 524/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:33.860976 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5a47-e326-4e68-9789-e7d7dea00e12 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:33.861322 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5a47-e326-4e68-9789-e7d7dea00e12 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:33.861619 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5a47-e326-4e68-9789-e7d7dea00e12 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:33.863137 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 519/1043] 10.222.0.13 () {54 vars in 961 bytes} [Mon Oct 17 05:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:33.868849 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:33.871953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:33.871953 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:33.951645 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] MFA Rules not processed for user `231350e78b2740a4b0aef761f7baa4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:33.952998 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:34.080810 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 525/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Oct 17 05:33:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:34.083410 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8fbe-fdaa-4603-b08f-b77f726a7c48 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:34.083699 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8fbe-fdaa-4603-b08f-b77f726a7c48 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:34.084037 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8fbe-fdaa-4603-b08f-b77f726a7c48 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:34.085516 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 520/1045] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:33:34.089560 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:33:34.090870 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b6da0-58f7-49aa-9f50-89a5aa90dafa None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:34.091254 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b6da0-58f7-49aa-9f50-89a5aa90dafa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:34.091674 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b6da0-58f7-49aa-9f50-89a5aa90dafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:34.100366 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd9bd2-a58a-43d4-9c46-7fdfa5721d19 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:33:34.143104 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:34.168725 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-280b6da0-58f7-49aa-9f50-89a5aa90dafa None None] MFA Rules not processed for user `d7f0e8d5f450484e921b5c820f033ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:33:34.170111 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-280b6da0-58f7-49aa-9f50-89a5aa90dafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:34.222073 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '231350e78b2740a4b0aef761f7baa4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6805b8cfa1834ce7a35ab033d20909b6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:33:34.222688 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:33:34.222965 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:33:34.223247 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:33:34.224928 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:33:34.279369 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 521/1046] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:33:34.338526 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:33:34.340232 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392997c-64c2-4fd4-83be-747de01b4bf8 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:33:34.361723 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 526/1047] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 05:33:44.277673 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:44.358811 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=79 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:33:54.862687 dv-cinder-861566-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Oct 17 05:33:54.863433 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: Mon Oct 17 05:33:54 2022 - graceful shutdown triggered... Oct 17 05:33:54.863891 dv-cinder-861566-1-iscsi devstack@keystone.service[81582]: Gracefully killing worker 2 (pid: 81582)... Oct 17 05:33:54.864240 dv-cinder-861566-1-iscsi devstack@keystone.service[81581]: Gracefully killing worker 1 (pid: 81581)... Oct 17 05:33:55.864555 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: worker 1 buried after 1 seconds Oct 17 05:33:55.864974 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: worker 2 buried after 1 seconds Oct 17 05:33:55.864974 dv-cinder-861566-1-iscsi devstack@keystone.service[81570]: goodbye to uWSGI. Oct 17 05:33:55.866122 dv-cinder-861566-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Oct 17 05:33:55.866652 dv-cinder-861566-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Oct 17 05:39:13.873285 dv-cinder-861566-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 05:39:13.885898 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 05:39:13 2022] *** Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: nodename: dv-cinder-861566-1-iscsi Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: machine: x86_64 Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: clock source: unix Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: pcre jit disabled Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: detected number of CPU cores: 4 Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: current working directory: / Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: detected binary path: /usr/bin/uwsgi-core Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: your processes number limit is 47680 Oct 17 05:39:13.888731 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: your memory page size is 4096 bytes Oct 17 05:39:13.891124 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: detected max file descriptor number: 2048 Oct 17 05:39:13.891124 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: lock engine: pthread robust mutexes Oct 17 05:39:13.891124 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: thunder lock: enabled Oct 17 05:39:13.891124 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 05:39:13.891124 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 05:39:13.928428 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: Python main interpreter initialized at 0x561186d149c0 Oct 17 05:39:13.928428 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: python threads support enabled Oct 17 05:39:13.928428 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: your server socket listen backlog is limited to 100 connections Oct 17 05:39:13.928428 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: your mercy for graceful operations on workers is 90 seconds Oct 17 05:39:13.928796 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 05:39:13.928879 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: *** Operational MODE: preforking *** Oct 17 05:39:13.929123 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: *** uWSGI is running in multiple interpreter mode *** Oct 17 05:39:13.929123 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: spawned uWSGI master process (pid: 153106) Oct 17 05:39:13.929318 dv-cinder-861566-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 05:39:13.933046 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: spawned uWSGI worker 1 (pid: 153118, cores: 1) Oct 17 05:39:13.933046 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: spawned uWSGI worker 2 (pid: 153119, cores: 1) Oct 17 05:39:13.933046 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 05:39:18.299466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:39:18.299466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: from cryptography.utils import int_from_bytes Oct 17 05:39:18.300367 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:39:18.300367 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: from cryptography.utils import int_from_bytes Oct 17 05:39:18.300367 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:39:18.300367 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: from cryptography.utils import int_from_bytes Oct 17 05:39:18.301015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:39:18.301015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: from cryptography.utils import int_from_bytes Oct 17 05:39:18.350605 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 05:39:18.350782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 05:39:18.350883 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 05:39:18.350986 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 05:39:18.351079 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] command line args: [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 05:39:18.351165 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] command line args: [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 05:39:18.351268 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 05:39:18.351502 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 05:39:18.351601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ================================================================================ {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 05:39:18.351706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ================================================================================ {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 05:39:18.351799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] admin_token = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.351884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] admin_token = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.351974 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] config_dir = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352077 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] config_dir = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352170 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352376 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] config_source = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352475 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] config_source = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352567 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] debug = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352658 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] debug = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.352756 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353013 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353172 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353293 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353412 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] insecure_debug = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353548 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] insecure_debug = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353801 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.353920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354168 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354303 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354422 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_config_append = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354535 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_config_append = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354764 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.354883 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_dir = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_dir = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_file = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_file = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355702 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.355973 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356057 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356162 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356262 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356356 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356773 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.356991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357278 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357383 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357482 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357840 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.357944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] notification_format = cadf {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] notification_format = cadf {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358322 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358560 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358681 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] publish_errors = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] publish_errors = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.358969 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359081 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359212 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359436 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359670 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.359917 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360165 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] standard_threads = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] standard_threads = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360407 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] strict_password_check = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] strict_password_check = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360661 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] use_eventlog = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.360959 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] use_eventlog = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] use_journal = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361150 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] use_journal = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361244 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] use_json = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361342 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] use_json = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] use_stderr = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361527 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] use_stderr = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361619 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] use_syslog = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] use_syslog = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] watch_log_file = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.361907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] watch_log_file = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:39:18.362013 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362100 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362506 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362612 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362754 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.362853 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363408 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363641 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.363908 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.364056 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.364149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.364316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.364416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365150 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365264 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365364 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365679 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365825 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.365942 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366339 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366433 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.366945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367116 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367224 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367510 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367744 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.367908 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368006 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368177 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368280 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368697 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.368939 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369180 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369663 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369766 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.369920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370012 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370280 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370649 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.370980 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371236 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371675 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.371916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372402 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372501 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.372972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373209 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373356 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373456 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373698 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.373971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374204 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374448 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374579 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374823 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.374917 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375511 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.375914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376059 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376304 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376396 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376545 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376653 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.376897 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377140 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377289 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377396 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377780 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.377879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378020 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378286 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378378 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378533 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378772 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.378870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379562 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379655 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379810 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.379902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.380066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.380158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.380887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.380974 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381138 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381237 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381381 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381480 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.enabled = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.381982 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.enabled = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382132 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382379 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382631 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382735 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382876 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.382966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383511 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383649 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.383896 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384386 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384754 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.384908 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385167 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385274 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.385916 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386242 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386397 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.386976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387127 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387405 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387492 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387626 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.387969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388119 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388456 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388625 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.388979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389136 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389756 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.external = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.389986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.external = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.mapped = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390226 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.mapped = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390534 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390684 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390775 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.390923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.password = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391013 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.password = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] auth.token = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391270 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] auth.token = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391433 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] catalog.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] catalog.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.391947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392038 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392172 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392271 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392531 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392792 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.392933 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393436 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393527 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.393955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394205 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394307 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394459 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394703 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394793 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.394959 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395050 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.395878 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396220 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396336 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396607 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396698 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396859 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.396949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397222 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397366 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397456 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.397963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398121 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398212 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398614 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398860 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.398951 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399121 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399654 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399901 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.399993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400155 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400265 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400493 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400760 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.400911 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401655 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.401895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402140 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402230 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402397 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402656 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.402987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403137 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403245 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403492 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403752 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.403920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404442 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.404980 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405325 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405476 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405721 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405819 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.405969 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406067 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406324 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406457 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406707 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406803 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.406958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407049 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407366 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407479 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407830 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.407993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408594 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.408987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409079 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409326 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.password = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409564 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.password = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.409963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410303 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410458 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410548 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.410914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411379 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411475 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.411968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412400 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.412949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413169 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413255 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413345 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413595 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413868 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.413962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414111 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414203 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414630 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414721 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414888 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.414978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415388 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415633 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415740 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.415889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416122 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416214 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416362 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416619 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.416980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.417994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418326 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418432 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418607 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418690 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418854 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.418964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419373 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419635 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419883 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.419977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.420147 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] policy.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.420246 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] policy.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.420388 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.420497 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.420907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421001 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421154 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421246 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421401 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] receipt.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421505 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] receipt.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421644 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421742 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421900 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.421999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422148 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422250 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422728 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422883 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.422983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423242 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423420 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423534 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423833 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.423923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] revoke.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] revoke.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.424904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425152 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] role.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] role.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] role.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425502 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] role.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425657 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] role.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] role.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425897 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] role.list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.425996 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] role.list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426151 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426242 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426766 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.426922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.427540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.427728 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.427821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.427988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428099 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428341 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428499 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428848 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.428995 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.429094 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.429254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.429516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.429622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.429777 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.429870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430779 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.430938 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431346 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.431975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432079 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432486 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432588 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432834 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.432972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433311 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433452 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433552 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.433979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434070 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434227 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.434956 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435060 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435329 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435477 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435586 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.provider = fernet {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435727 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.provider = fernet {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.435972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436329 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436486 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436587 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436820 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.436982 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] trust.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437482 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437573 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] trust.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437728 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437827 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.437973 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438077 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438216 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438564 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438836 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.438981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439074 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439378 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.connection = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.439890 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.connection = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440294 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440397 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440631 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.440878 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441122 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441275 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441518 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.441855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442010 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442102 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442355 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442499 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442795 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.442889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443147 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.443910 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.444052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.444143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.444237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=153118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 05:39:18.444322 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:39:18.444458 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=153119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 05:39:18.473620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.473801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.474439 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.474565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.474673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.474798 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.483056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.483185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.512073 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.512256 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.512354 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.512495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.566261 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.566827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.615357 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.615598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.616107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.616351 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.773437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.773650 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.773888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774649 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.774795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.775023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.775247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799087 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799231 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799349 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799521 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.799864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800147 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800266 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800365 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800581 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.800714 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.812379 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.812805 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.825485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.825676 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.825873 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.825979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826347 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826453 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826650 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826780 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=153119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.826875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.827006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.827230 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=153118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:39:18.844217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.844432 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:39:18.844676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.844771 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.844899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:39:18.845007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:39:18.845243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.845344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.845458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:39:18.845551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.845747 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.845842 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.846022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.846114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.846295 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.846568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.855764 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.856054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.856308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.856403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.856602 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.856715 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.856889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.857188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.863646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.864251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.867087 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.867404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.867700 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.867800 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.867994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868088 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868378 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868646 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.868941 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869122 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869537 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869788 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.869965 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.870064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.870252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.870536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.893275 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.893910 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.894758 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.895434 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.895574 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.895754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.900285 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.900467 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.900567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.900762 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.900869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.901059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.907227 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.907461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.907561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.907761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.913198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.913447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.913547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.913753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.913857 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914051 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914165 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914433 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.914906 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.915014 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.915186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.915283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.915494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.931540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.931831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.931928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.932157 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.936006 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.936122 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.939266 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.939379 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.939602 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.939707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.943929 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.944027 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.947133 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.947411 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.950463 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.950667 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.950777 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.950979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.951075 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.951277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.951405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.951607 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.951720 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.951914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.952012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.952213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.968401 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.968543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.968705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.968843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.968987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.969124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.969265 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.969399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.976752 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.976880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.978326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.978441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.979904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.980092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.980187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.980378 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.982955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.983057 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.983255 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.983441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.983584 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.983774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.983865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.983952 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.984270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.984366 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.995479 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.995580 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:18.995774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.995872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.996065 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.996175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.996353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.996449 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.996747 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.996854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.997049 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.997161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.997341 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:18.997445 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.012842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.012950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.015789 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.015894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.018747 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.018848 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.020307 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.020430 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.022343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.022447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.022633 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.022735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.023042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.023150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.029459 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.029571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.030932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.031035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.033862 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.033977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.034272 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.034369 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.034574 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.034680 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.034868 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.034971 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.042816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.043062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.043175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.043392 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.043490 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.043689 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.050629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=153118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.050860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=153119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:39:19.050967 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.051158 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.051254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.051482 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.051590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.051773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.051887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.052587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.052693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.052871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.052965 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.053154 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.053257 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.053446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.053540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.053722 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.053826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054012 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054108 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054436 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054584 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.054952 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=153118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.055145 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=153119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:39:19.089680 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561186d149c0 pid: 153118 (default app) Oct 17 05:39:19.089831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561186d149c0 pid: 153119 (default app) Oct 17 05:39:19.093066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52202b-bca2-4f8f-a467-f2824c9e7b79 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:19.093368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52202b-bca2-4f8f-a467-f2824c9e7b79 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:19.093654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52202b-bca2-4f8f-a467-f2824c9e7b79 None None] PATH_INFO: `/v3` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:19.097880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon Oct 17 05:39:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:26.633542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c45f3-ba67-466f-8880-704b978434bc None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:26.634088 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c45f3-ba67-466f-8880-704b978434bc None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:26.634088 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c45f3-ba67-466f-8880-704b978434bc None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:26.638462 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:26.643680 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:26.644090 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:26.644245 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:26.715376 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: INFO dbcounter [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] Registered counter for database keystone Oct 17 05:39:26.733816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG oslo_db.sqlalchemy.engines [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=153118) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 05:39:26.740200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writer thread running {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 05:39:26.921177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG passlib.handlers.bcrypt [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=153118) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 05:39:26.921479 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG passlib.handlers.bcrypt [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=153118) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 05:39:26.979114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:26.980729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:27.106467 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 05:39:27.371578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c45f3-ba67-466f-8880-704b978434bc None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:27.441651 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: INFO dbcounter [None req-429c45f3-ba67-466f-8880-704b978434bc None None] Registered counter for database keystone Oct 17 05:39:27.463433 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG oslo_db.sqlalchemy.engines [None req-429c45f3-ba67-466f-8880-704b978434bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=153119) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 05:39:27.469802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writer thread running {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 05:39:27.587934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c45f3-ba67-466f-8880-704b978434bc None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:27.592268 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:27.592892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:27.593179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:27.593457 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:27.602210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:27.652934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 05:39:27.653507 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca217e4-762b-479c-8748-d8f5c166681d admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:27.663051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Oct 17 05:39:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 05:39:27.669048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:27.685913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7969703-6c95-46af-ad20-f7d0575498c9 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:27.690359 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:27.691491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:27.691779 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:27.692054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] PATH_INFO: `/v3/projects/be40c9d78f744db094612526ea19176d` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:27.694176 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] RBAC: Authorizing `identity:get_project(project_id=be40c9d78f744db094612526ea19176d)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:27.745950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: WARNING keystone.common.rbac_enforcer.enforcer [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 05:39:27.746305 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b22bb6-63a4-42d7-8218-54e2388073fd admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:27.750782 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:39:27 2022] GET /identity/v3/projects/be40c9d78f744db094612526ea19176d => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:29.871176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6160fd-ebb7-4e87-b6e9-bd031f1c6450 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:29.871794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6160fd-ebb7-4e87-b6e9-bd031f1c6450 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:29.871794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6160fd-ebb7-4e87-b6e9-bd031f1c6450 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:29.875861 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:29.881337 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:29.881770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:29.881914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:29.901911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:29.903472 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:29.911996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:39:30.180321 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6160fd-ebb7-4e87-b6e9-bd031f1c6450 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:30.192977 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6160fd-ebb7-4e87-b6e9-bd031f1c6450 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:30.197275 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ad6b5-1bc4-4e9c-b3d3-607dcb1ecd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:30.197873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad6b5-1bc4-4e9c-b3d3-607dcb1ecd8b admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:30.198159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad6b5-1bc4-4e9c-b3d3-607dcb1ecd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:30.198440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ad6b5-1bc4-4e9c-b3d3-607dcb1ecd8b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:30.205966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad6b5-1bc4-4e9c-b3d3-607dcb1ecd8b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:30.207842 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ad6b5-1bc4-4e9c-b3d3-607dcb1ecd8b admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:30.216506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Oct 17 05:39:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 05:39:30.221634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:30.230829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bca64-7090-4f65-b459-ba752c1cf06e None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:30.235217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b6b39-a8c9-46a4-bac5-5f5620cfdd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:30.235889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6b39-a8c9-46a4-bac5-5f5620cfdd62 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:30.236181 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6b39-a8c9-46a4-bac5-5f5620cfdd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:30.236469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6b39-a8c9-46a4-bac5-5f5620cfdd62 admin admin] PATH_INFO: `/v3/users/437c2d620e4648bb9b558b4098744fa0` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:30.237798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b6b39-a8c9-46a4-bac5-5f5620cfdd62 admin admin] RBAC: Authorizing `identity:get_user(user_id=437c2d620e4648bb9b558b4098744fa0)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:30.240202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b6b39-a8c9-46a4-bac5-5f5620cfdd62 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:30.241737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon Oct 17 05:39:30 2022] GET /identity/v3/users/437c2d620e4648bb9b558b4098744fa0 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:32.435341 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5f547-1a51-4bb5-ab02-20c986e5678c None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:32.435926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5f547-1a51-4bb5-ab02-20c986e5678c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:32.435926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5f547-1a51-4bb5-ab02-20c986e5678c None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:32.437233 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:39:32.443329 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:32.443767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:32.443902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:32.461794 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:32.462976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:32.481836 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c83f2a-363c-479a-9801-391249de55f2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:32.482125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c83f2a-363c-479a-9801-391249de55f2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:32.482407 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c83f2a-363c-479a-9801-391249de55f2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:32.483772 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:39:32.489627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:32.489911 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:32.490188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:32.532243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG passlib.handlers.bcrypt [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=153119) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 05:39:32.532646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG passlib.handlers.bcrypt [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=153119) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 05:39:32.544691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:32.582164 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:32.583840 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:32.683744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:32.815079 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:32.839667 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:32.919642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085b58b-90bc-4e0c-88dd-15c7eaec8feb None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:32.951005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:32.953574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:32.954221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:32.954504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:32.954790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:32.966759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:32.975832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:32.977114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:32.978199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:39:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:32.982636 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:33.056470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f263f9b-3a67-4697-976f-66319f2bde29 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:33.066194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:33.084642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87fd6c-5258-41f2-a175-6d63ef9be36b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.095421 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.096293 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.096588 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.096873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.097861 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.099980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.119164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.119811 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.120101 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.120381 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.121060 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:33.121375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.122779 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.134527 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:39:33.140125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:33.157144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.sql.core [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:39:33.157144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:39:33.157144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [parameters: {'id': '8a3c6f5719f04319a3b54dd74140a82c', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:39:33.157144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=153119) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:39:33.176453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result.read() Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:39:33.177566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application yield resource Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise self.value Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application session.commit() Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:39:33.179125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:39:33.180375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:39:33.181590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result.read() Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:39:33.182877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [parameters: {'id': '8a3c6f5719f04319a3b54dd74140a82c', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:39:33.184192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:33.185271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 8/16] 10.222.0.13 () {60 vars in 1137 bytes} [Mon Oct 17 05:39:32 2022] POST /identity/v3/domains => generated 190 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:39:33.186761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:33.242509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c8c52-0f30-4a82-944c-347e397d1b07 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.277190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.277822 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.278107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.278389 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] PATH_INFO: `/v3/users/437c2d620e4648bb9b558b4098744fa0` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.279557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] RBAC: Authorizing `identity:get_user(user_id=437c2d620e4648bb9b558b4098744fa0)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.281015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.282766 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:33 2022] GET /identity/v3/users/437c2d620e4648bb9b558b4098744fa0 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:39:33.283733 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe3b0-2d44-45da-bc8b-02ab94bfef6a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.288169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:33.303287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:33.318682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.319278 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.319601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.319889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.338186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.339703 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:39:33.356447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:33.357627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:39:33.358628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:39:33.358628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:39:33.358628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:33.358628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:33.363326 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:33.385416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:33.387057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8627-8b82-4f6f-9847-0dbe38a910e3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.422401 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.423054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.423381 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.423672 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.424829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.426306 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.428098 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:39:33.433618 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:33.465555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60529f-0b8f-4cde-9cd9-f2e0c3620113 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.501736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.502348 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.502644 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.502921 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.503613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:33.503929 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.505819 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff509ff-a12b-42f2-a5bc-4e4a846f187e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.514786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:33.523506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:39:33.534668 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b357ee8-df19-4cef-8cb4-e3e7fc5dab95 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:33.572640 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70a647-2892-4ff7-895e-339b9d4f9787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:33.573244 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a647-2892-4ff7-895e-339b9d4f9787 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:33.573528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a647-2892-4ff7-895e-339b9d4f9787 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:33.573809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a647-2892-4ff7-895e-339b9d4f9787 None admin] PATH_INFO: `/v3/domains/default/users/437c2d620e4648bb9b558b4098744fa0/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:33.575911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70a647-2892-4ff7-895e-339b9d4f9787 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=437c2d620e4648bb9b558b4098744fa0, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:33.577426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70a647-2892-4ff7-895e-339b9d4f9787 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:33.754239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 11/21] 10.222.0.13 () {58 vars in 1361 bytes} [Mon Oct 17 05:39:33 2022] PUT /identity/v3/domains/default/users/437c2d620e4648bb9b558b4098744fa0/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:39:36.013509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c9845-59ec-48aa-a458-83fd9a5db181 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.014066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c9845-59ec-48aa-a458-83fd9a5db181 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.014066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c9845-59ec-48aa-a458-83fd9a5db181 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.015494 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:36.025536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea3fe-59fe-4b3f-affb-8f6f5a9c19d2 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.025998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea3fe-59fe-4b3f-affb-8f6f5a9c19d2 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.026107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea3fe-59fe-4b3f-affb-8f6f5a9c19d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.060526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-38dea3fe-59fe-4b3f-affb-8f6f5a9c19d2 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:36.062033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-38dea3fe-59fe-4b3f-affb-8f6f5a9c19d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:36.157664 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:36.483653 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfaddd-8aca-4cd3-9ced-e5264f9046c5 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.484051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfaddd-8aca-4cd3-9ced-e5264f9046c5 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.484266 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfaddd-8aca-4cd3-9ced-e5264f9046c5 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.485827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:36.491432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6454677-543a-4a25-8e11-0357a5532dc7 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.491734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6454677-543a-4a25-8e11-0357a5532dc7 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.492024 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6454677-543a-4a25-8e11-0357a5532dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.516339 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-b6454677-543a-4a25-8e11-0357a5532dc7 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:36.517566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-b6454677-543a-4a25-8e11-0357a5532dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:36.601989 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:36.606923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:36.697207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfaddd-8aca-4cd3-9ced-e5264f9046c5 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:36.710049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2438e70-daa3-4b43-8df3-5156fa0c13ba None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.710350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2438e70-daa3-4b43-8df3-5156fa0c13ba None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.710623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2438e70-daa3-4b43-8df3-5156fa0c13ba None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.719609 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 14/26] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:36.721045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253508b2-cb8d-455b-8330-a573d8de09cf None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.721564 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253508b2-cb8d-455b-8330-a573d8de09cf None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.722074 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253508b2-cb8d-455b-8330-a573d8de09cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.750396 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-253508b2-cb8d-455b-8330-a573d8de09cf None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:36.751774 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-253508b2-cb8d-455b-8330-a573d8de09cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:36.855066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 15/27] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:36.857007 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbb587-fd84-4467-8562-48258e63c3d7 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.857544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbb587-fd84-4467-8562-48258e63c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.858154 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbb587-fd84-4467-8562-48258e63c3d7 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.864423 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 16/28] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:36.866298 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:36.866835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:36.867414 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:36.888072 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:36.889732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:36.900522 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:36.964019 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfaddd-8aca-4cd3-9ced-e5264f9046c5 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:36.971425 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 17/29] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:37.028967 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:37.029520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.029803 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.030089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.050866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:37.052936 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:39:37.071374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:37.072463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:39:37.074096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:39:37.074096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:39:37.074096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:37.074096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 13/30] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:37.078344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:37.079927 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:37.161333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:37.207855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0280e7-b9c8-4b7b-8188-0a103290d04a None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:37.246210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:37.247341 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.247861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.248360 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.249231 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:37.249772 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:37.251601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:37.272002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 18/31] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:37.280350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:37.293411 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7827-003d-44c4-88ad-bfce4aed752b None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:37.351984 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:37.358085 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:37.358888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.359374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.359826 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.360668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:37.362308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:37.423382 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ca902-4870-4122-808e-20a5d9768b53 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:37.445372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:37.460916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:37.461638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.461938 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.462223 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.462906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:37.464559 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:37.583256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 19/32] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:39:37 2022] POST /identity/v3/projects => generated 301 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:37.584681 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:37.607259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 14/33] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:39:37 2022] POST /identity/v3/roles => generated 225 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:39:37.608248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:37.809494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b6f5e-0b0f-49c9-b0e2-abab35d2ae65 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:37.815754 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6bcda-f05a-4e0f-bf6d-3fb7b507d7d8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:37.854082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c594599-7820-4ca8-83b6-0c3f45c80160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:37.854715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594599-7820-4ca8-83b6-0c3f45c80160 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.855010 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594599-7820-4ca8-83b6-0c3f45c80160 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.855317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594599-7820-4ca8-83b6-0c3f45c80160 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.856111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c594599-7820-4ca8-83b6-0c3f45c80160 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:37.857425 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c594599-7820-4ca8-83b6-0c3f45c80160 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:37.864471 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c61de-348f-4e31-8466-56fb3309cb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:37.865264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c61de-348f-4e31-8466-56fb3309cb1e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.865556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c61de-348f-4e31-8466-56fb3309cb1e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.865892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c61de-348f-4e31-8466-56fb3309cb1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.878737 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c61de-348f-4e31-8466-56fb3309cb1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ed0c61de-348f-4e31-8466-56fb3309cb1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:37.885601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:37.886636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:37.887718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:37.891335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 15/34] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:39:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:37.891335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc44ba-e2d7-4a97-a8e7-d13119d339b1 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.891335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc44ba-e2d7-4a97-a8e7-d13119d339b1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.891335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc44ba-e2d7-4a97-a8e7-d13119d339b1 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.892238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 16/35] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:37.893077 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b5678-7aa6-4aaa-bcb8-33bf7e63ab32 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.893370 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b5678-7aa6-4aaa-bcb8-33bf7e63ab32 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.894177 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b5678-7aa6-4aaa-bcb8-33bf7e63ab32 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.895781 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 17/36] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:37.896441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b5678-7aa6-4aaa-bcb8-33bf7e63ab32 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:37.924978 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 20/37] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:39:37 2022] POST /identity/v3/roles => generated 219 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:37.926691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc8968-dcfc-4957-817b-505bf664458a None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:37.927145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc8968-dcfc-4957-817b-505bf664458a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:37.927516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc8968-dcfc-4957-817b-505bf664458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:37.947182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-6dbc8968-dcfc-4957-817b-505bf664458a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:37.948543 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-6dbc8968-dcfc-4957-817b-505bf664458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:37.966081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:37.978223 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:38.024385 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b5678-7aa6-4aaa-bcb8-33bf7e63ab32 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.027731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 21/38] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:38.028398 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.029093 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.030001 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.051755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:38.053202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:38.074838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.076591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.077062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.078136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.079031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:38.079812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.082184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.099807 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 18/39] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:38.107983 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:38.231190 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72943b1-3836-4ea3-b2d6-f079cb9860ea None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.247333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 22/40] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:38.247818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:38.267063 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.267803 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.268100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.268448 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] PATH_INFO: `/v3/users/437c2d620e4648bb9b558b4098744fa0` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.270698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] RBAC: Authorizing `identity:get_user(user_id=437c2d620e4648bb9b558b4098744fa0)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.272240 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.274003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 19/41] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:38 2022] GET /identity/v3/users/437c2d620e4648bb9b558b4098744fa0 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:38.279507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:38.300728 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:38.376510 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:38.385957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19ba7e-5b3d-4b2e-afe6-e2b3dc1c96ca None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.411874 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212c787-7206-4c31-a34e-b94b81d51f74 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.422881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.423875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.424338 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.424659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.425816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.427836 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.429437 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 23/42] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:38.435050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:38.450345 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.451274 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.451746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.452226 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.453172 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.454858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.534654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e8481-116a-4ac6-9fed-cbec5928276d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.552952 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 20/43] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:39:38 2022] POST /identity/v3/roles => generated 223 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:39:38.553632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:38.577446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.578829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.579130 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.579788 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.580585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.domain.id=default, user.id=437c2d620e4648bb9b558b4098744fa0) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:38.581179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.584135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.604249 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 24/44] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Oct 17 05:39:38 2022] GET /identity/v3/role_assignments?user.id=437c2d620e4648bb9b558b4098744fa0&scope.domain.id=default&role.id=df4da7cff6734799a09e0b770b60845f => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:39:38.604476 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:38.675107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faa0cd-e8f0-4cba-96af-add4c103f5a5 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.715872 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd60e44-bd5b-4b9d-8511-7aded1fa612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.716549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd60e44-bd5b-4b9d-8511-7aded1fa612e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.716840 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd60e44-bd5b-4b9d-8511-7aded1fa612e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.717123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd60e44-bd5b-4b9d-8511-7aded1fa612e None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.718467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd60e44-bd5b-4b9d-8511-7aded1fa612e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.720024 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd60e44-bd5b-4b9d-8511-7aded1fa612e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.721937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 21/45] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:38.728406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:38.762422 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:38.850714 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603efa5-e277-4e2d-bad7-e165b9c48840 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:38.884553 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78997b-c5c2-4a91-85c8-1472d55423c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:38.885169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78997b-c5c2-4a91-85c8-1472d55423c3 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:38.885456 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78997b-c5c2-4a91-85c8-1472d55423c3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:38.885735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78997b-c5c2-4a91-85c8-1472d55423c3 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:38.886403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78997b-c5c2-4a91-85c8-1472d55423c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:38.887906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78997b-c5c2-4a91-85c8-1472d55423c3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:38.991691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 25/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:39:38 2022] POST /identity/v3/projects => generated 312 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:41.124247 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588059-91fa-49a4-a9c9-e55bb79fc9c4 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:41.130706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588059-91fa-49a4-a9c9-e55bb79fc9c4 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:41.130706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588059-91fa-49a4-a9c9-e55bb79fc9c4 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:41.130706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 22/47] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:41.132032 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:41.132432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:41.132610 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:41.151442 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:41.153588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:41.218517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 26/48] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:41.487866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68588059-91fa-49a4-a9c9-e55bb79fc9c4 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:41.589510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68588059-91fa-49a4-a9c9-e55bb79fc9c4 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:41.599458 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:41.622319 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a270f-840a-4386-80a0-6cce3a43603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:41.622950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a270f-840a-4386-80a0-6cce3a43603a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:41.623241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a270f-840a-4386-80a0-6cce3a43603a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:41.623555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a270f-840a-4386-80a0-6cce3a43603a None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:41.624684 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a270f-840a-4386-80a0-6cce3a43603a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:41.626252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a270f-840a-4386-80a0-6cce3a43603a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:41.628183 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 23/49] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:41.633787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:41.641825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:41.732345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af7fbb-5e79-48b3-ac93-898db15f86a9 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:41.765960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db5119-fc4c-48ff-ae16-b340d12199e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:41.766509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5119-fc4c-48ff-ae16-b340d12199e1 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:41.766799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5119-fc4c-48ff-ae16-b340d12199e1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:41.767082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5119-fc4c-48ff-ae16-b340d12199e1 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:41.767769 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db5119-fc4c-48ff-ae16-b340d12199e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:41.769234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db5119-fc4c-48ff-ae16-b340d12199e1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:41.854844 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 27/50] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Oct 17 05:39:41 2022] POST /identity/v3/projects => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:43.982858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b7bd-4c9e-4cb2-b561-fb4fb6152842 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:43.983475 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b7bd-4c9e-4cb2-b561-fb4fb6152842 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:43.983475 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b7bd-4c9e-4cb2-b561-fb4fb6152842 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:43.984817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 24/51] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:43.990562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:43.991007 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:43.991137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:44.009913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:44.011128 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:44.063346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:44.076404 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 28/52] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:44.347350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789b7bd-4c9e-4cb2-b561-fb4fb6152842 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:44.448260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789b7bd-4c9e-4cb2-b561-fb4fb6152842 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:44.480498 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8983f-abc4-4b21-baaf-67b9c23bbc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:44.481119 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8983f-abc4-4b21-baaf-67b9c23bbc55 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:44.481410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8983f-abc4-4b21-baaf-67b9c23bbc55 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:44.481693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8983f-abc4-4b21-baaf-67b9c23bbc55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:44.482810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8983f-abc4-4b21-baaf-67b9c23bbc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:44.484277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8983f-abc4-4b21-baaf-67b9c23bbc55 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:44.485817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 25/53] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:44.491907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:44.589188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6035-0f7f-483d-a35f-5ae79e145378 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:44.622558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f39238-e3fc-4c50-9678-b099458d8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:44.623107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f39238-e3fc-4c50-9678-b099458d8e7b None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:44.623433 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f39238-e3fc-4c50-9678-b099458d8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:44.623724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f39238-e3fc-4c50-9678-b099458d8e7b None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:44.624386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f39238-e3fc-4c50-9678-b099458d8e7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:44.625843 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f39238-e3fc-4c50-9678-b099458d8e7b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:44.717083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 29/54] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:39:44 2022] POST /identity/v3/users => generated 275 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:47.934068 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8e34-0c57-4ba9-9ce4-53ffe97452ae None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:47.943766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8e34-0c57-4ba9-9ce4-53ffe97452ae None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:47.943766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8e34-0c57-4ba9-9ce4-53ffe97452ae None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:47.943766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 26/55] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:47.944021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f854-6a17-4b14-805b-ce469eec80e8 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:47.944021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f854-6a17-4b14-805b-ce469eec80e8 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:47.944021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f854-6a17-4b14-805b-ce469eec80e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:47.968151 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-bba0f854-6a17-4b14-805b-ce469eec80e8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:47.974760 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-bba0f854-6a17-4b14-805b-ce469eec80e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:48.019034 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:48.058444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 30/56] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:48.210072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b384-2de9-420b-9e5c-85a50c20e9df None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.210757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b384-2de9-420b-9e5c-85a50c20e9df None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.211257 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b384-2de9-420b-9e5c-85a50c20e9df None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.212865 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 27/57] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:48.218102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.218399 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.218684 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.240301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:48.241509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:48.247971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de0b62-782b-4eb5-96b1-bb02b04fa75d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.248584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de0b62-782b-4eb5-96b1-bb02b04fa75d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.249083 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de0b62-782b-4eb5-96b1-bb02b04fa75d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.250654 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:39:48.256469 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d98082-c4de-4354-8dd3-dc080343b322 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.257563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d98082-c4de-4354-8dd3-dc080343b322 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.258088 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d98082-c4de-4354-8dd3-dc080343b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.291628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-88d98082-c4de-4354-8dd3-dc080343b322 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:48.293157 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-88d98082-c4de-4354-8dd3-dc080343b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:48.366140 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:48.379618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:48.402829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:48.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 29/60] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:48.451682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a436f-bb39-4b44-86db-245c20f4d5ea None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.451989 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a436f-bb39-4b44-86db-245c20f4d5ea None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.452252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a436f-bb39-4b44-86db-245c20f4d5ea None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.453593 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 30/61] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:48.463185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.463503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.463784 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.491342 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:48.492550 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:48.548998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6057e3-3dcc-4460-a9ce-1f3c29d38d84 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:48.567657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:48.585512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:48.586404 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.586850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.587330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.599290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:48.605691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:48.606880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:48.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:48.610125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 32/62] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:39:48.614148 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 31/63] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:48.616415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:48.645531 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:48.720700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcff023-1c85-4b28-95d8-852b257ff230 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:48.759351 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:48.760223 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.760693 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.761138 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.762023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:48.762592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:48.764013 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:48.774904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 33/64] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:39:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:48.780508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:48.800510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8b53-353d-4483-b695-e099e5736807 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:48.809162 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:48.824593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:48.839351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:48.840011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.840305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.840590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.851256 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:48.856488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:48.857570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:48.858510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:48.861238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 32/65] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:39:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:48.871517 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:48.900791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49a303-bbbb-4890-a8ad-d6d1c37006a8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:48.939069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:48.939759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:48.940050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:48.940331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:48.950928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:48.955914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:48.956933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 34/66] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:39:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:48.958405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:48.972291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cdf8a-a799-4422-a42a-63802b68fef7 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.006615 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.007367 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.007663 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.007947 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] PATH_INFO: `/v3/users/dc60bb94695043b38502ba313cbe55ad` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.049431 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.065163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d2dc4-d11c-4275-bc5d-9cf97b515841 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.102392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.103030 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.103330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.103618 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.104311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:49.104623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.106053 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4d3f2-1919-4ad1-88d4-64e71aa50fea None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.115156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 35/67] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.116630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3099586-a0f6-4ef7-ab0e-d50fc4aae550 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.116930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3099586-a0f6-4ef7-ab0e-d50fc4aae550 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.117210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3099586-a0f6-4ef7-ab0e-d50fc4aae550 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.118592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 36/68] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:49.122957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3099586-a0f6-4ef7-ab0e-d50fc4aae550 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.132484 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] RBAC: Authorizing `identity:get_user(user_id=dc60bb94695043b38502ba313cbe55ad)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.133960 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.135680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 33/69] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:48 2022] GET /identity/v3/users/dc60bb94695043b38502ba313cbe55ad => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.136456 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.162992 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.224693 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3099586-a0f6-4ef7-ab0e-d50fc4aae550 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.239020 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd894fb-a112-41c0-b7c0-d9402e0f20ee None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.257871 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.258504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.258791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.259121 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.259834 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:49.260196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.261590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.269941 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 37/70] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.273490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fed807-d9ed-4a11-8e29-71c27a88aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.274134 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fed807-d9ed-4a11-8e29-71c27a88aa82 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.274460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fed807-d9ed-4a11-8e29-71c27a88aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.274747 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fed807-d9ed-4a11-8e29-71c27a88aa82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.275274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.286334 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fed807-d9ed-4a11-8e29-71c27a88aa82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:49.287149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-22fed807-d9ed-4a11-8e29-71c27a88aa82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:49.291235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:49.292404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 34/71] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:49.298466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.299381 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.299381 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.312289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:49.313514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:49.375819 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340914-9396-4ddd-8751-e8cbbcd86448 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.380509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 35/72] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:49.381341 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.409896 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.410258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.410906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.411197 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.411526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] PATH_INFO: `/v3/users/437c2d620e4648bb9b558b4098744fa0` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.412672 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] RBAC: Authorizing `identity:get_user(user_id=437c2d620e4648bb9b558b4098744fa0)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.414147 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.415773 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 38/73] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/users/437c2d620e4648bb9b558b4098744fa0 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.416544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.499336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6056e-3290-4911-b1b0-da99c936c516 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.503923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.530191 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef1ce-9385-4ae7-8461-a66d9cec3134 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.535949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.536602 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.536900 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.537174 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.538338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.539838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.546841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 36/74] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.546841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.575046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.575754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.576048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.576481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] PATH_INFO: `/v3/users/dc60bb94695043b38502ba313cbe55ad` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.577683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] RBAC: Authorizing `identity:get_user(user_id=dc60bb94695043b38502ba313cbe55ad)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.579101 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.580902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 39/75] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/users/dc60bb94695043b38502ba313cbe55ad => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.581561 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.636504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.656514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37979-3c7d-4822-8b98-575b78d83302 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.687489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb04c89-df64-482e-8652-210b0ae4c488 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.693248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.693934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.694292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.694642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.695810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:49.696127 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.697886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.706488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 37/76] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.707056 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.725440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.726084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.726377 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.726663 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.727851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.729302 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.730959 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 40/77] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.736169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.751577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.814165 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6790081-2aad-461b-8738-e9b1c9637025 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.836447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945741b-e66d-4797-9a62-0da57c3030a9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:49.847491 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.848241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.848530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.848812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/dc60bb94695043b38502ba313cbe55ad/roles/153c5cb975cc4d2b868b1cf6ce5331cc` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.850767 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a1cdee355bb441ea1a53da3a457e54e, user_id=dc60bb94695043b38502ba313cbe55ad, role_id=153c5cb975cc4d2b868b1cf6ce5331cc)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.852250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.870172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:49.872285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:49.872285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:49.872285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:49.872643 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:49.874034 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:49.875737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 41/78] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:49.876575 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.924134 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 38/79] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:39:49 2022] PUT /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/dc60bb94695043b38502ba313cbe55ad/roles/153c5cb975cc4d2b868b1cf6ce5331cc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:39:49.924899 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:49.939366 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.968706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:49.991084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8635f-cbc6-43da-86a7-708ca74af5a1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.031969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.032937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.033482 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.033688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.034857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.036933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.043396 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 42/80] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:50.043396 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:50.058647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1848-6c62-4031-b125-e70d3f533f39 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.099843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.100508 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.100798 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.101083 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] PATH_INFO: `/v3/users/dc60bb94695043b38502ba313cbe55ad` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.103437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] RBAC: Authorizing `identity:get_user(user_id=dc60bb94695043b38502ba313cbe55ad)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.104673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.107335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 39/81] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:49 2022] GET /identity/v3/users/dc60bb94695043b38502ba313cbe55ad => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:50.107804 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:50.154042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d783-a5f6-44aa-9b18-949610c6adf2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.185935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:50.189993 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.190676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.190976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.191264 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/437c2d620e4648bb9b558b4098744fa0/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.193882 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a1cdee355bb441ea1a53da3a457e54e, user_id=437c2d620e4648bb9b558b4098744fa0, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.195384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.217837 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3058448-87b9-43c8-ac45-b440d65c92b4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.251990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.252656 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.253091 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.253328 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/dc60bb94695043b38502ba313cbe55ad/roles/c040847a15b148f893d961a9a578db29` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.255580 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a1cdee355bb441ea1a53da3a457e54e, user_id=dc60bb94695043b38502ba313cbe55ad, role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.257025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.299563 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 43/82] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:39:50 2022] PUT /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/437c2d620e4648bb9b558b4098744fa0/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:39:50.300434 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:50.314946 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:50.361755 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 40/83] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:39:50 2022] PUT /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/dc60bb94695043b38502ba313cbe55ad/roles/c040847a15b148f893d961a9a578db29 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:39:50.363075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:50.490107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720d3fa-d758-4591-9dc4-2ea8cb1d6a06 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.506166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdac5fc-0227-4647-8ab3-936b4c071503 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.547101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:50.555329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.556251 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.556563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.556920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] PATH_INFO: `/v3/projects/e2762dd6d69f4823859cf55d19460c63` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.558125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] RBAC: Authorizing `identity:get_project(project_id=e2762dd6d69f4823859cf55d19460c63)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.558293 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b485f2d-2bee-4ce2-9550-4cc53cd209c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.559097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b485f2d-2bee-4ce2-9550-4cc53cd209c3 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.559601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b485f2d-2bee-4ce2-9550-4cc53cd209c3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.563447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.563585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b485f2d-2bee-4ce2-9550-4cc53cd209c3 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.563585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b485f2d-2bee-4ce2-9550-4cc53cd209c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.563748 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 41/84] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:50 2022] GET /identity/v3/projects/e2762dd6d69f4823859cf55d19460c63 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:50.563836 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b485f2d-2bee-4ce2-9550-4cc53cd209c3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.571217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:50.683594 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f453e63-0e9c-47ce-a5f8-f0a80e4c23c4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:50.726720 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 44/85] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:39:50 2022] POST /identity/v3/users => generated 289 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:50.738974 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c97966-1e25-4b3f-af6f-a5a7352dbbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:50.740055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97966-1e25-4b3f-af6f-a5a7352dbbf9 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:50.740605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97966-1e25-4b3f-af6f-a5a7352dbbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:50.741110 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97966-1e25-4b3f-af6f-a5a7352dbbf9 None admin] PATH_INFO: `/v3/projects/e2762dd6d69f4823859cf55d19460c63/users/dc60bb94695043b38502ba313cbe55ad/roles/153c5cb975cc4d2b868b1cf6ce5331cc` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:50.743602 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c97966-1e25-4b3f-af6f-a5a7352dbbf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2762dd6d69f4823859cf55d19460c63, user_id=dc60bb94695043b38502ba313cbe55ad, role_id=153c5cb975cc4d2b868b1cf6ce5331cc)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:50.745408 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c97966-1e25-4b3f-af6f-a5a7352dbbf9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:50.815605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 42/86] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:39:50 2022] PUT /identity/v3/projects/e2762dd6d69f4823859cf55d19460c63/users/dc60bb94695043b38502ba313cbe55ad/roles/153c5cb975cc4d2b868b1cf6ce5331cc => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:39:52.329732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a412-0003-4794-b6ae-5683aefa51d6 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:52.330696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a412-0003-4794-b6ae-5683aefa51d6 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:52.331177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a412-0003-4794-b6ae-5683aefa51d6 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:52.333076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 45/87] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:52.338915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8ac19-83cd-437c-ac93-ffb66500f4ae None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:52.339788 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8ac19-83cd-437c-ac93-ffb66500f4ae None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:52.340263 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8ac19-83cd-437c-ac93-ffb66500f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:52.368335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-2ce8ac19-83cd-437c-ac93-ffb66500f4ae None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:52.369854 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-2ce8ac19-83cd-437c-ac93-ffb66500f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:52.451387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:52.463684 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 43/88] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:53.003121 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325a412-0003-4794-b6ae-5683aefa51d6 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:53.110203 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:53.148336 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325a412-0003-4794-b6ae-5683aefa51d6 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:53.152380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49199195-ebd6-42e4-880d-5eb067ae0aa4 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.152380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49199195-ebd6-42e4-880d-5eb067ae0aa4 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.152380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49199195-ebd6-42e4-880d-5eb067ae0aa4 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.154016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 44/89] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:53.159174 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfd396-88f0-4d0c-86e0-fbb228d842af None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.159512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfd396-88f0-4d0c-86e0-fbb228d842af None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.159801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfd396-88f0-4d0c-86e0-fbb228d842af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.191038 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-bbdfd396-88f0-4d0c-86e0-fbb228d842af None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:53.192849 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-bbdfd396-88f0-4d0c-86e0-fbb228d842af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:53.208570 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:53.209571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.210090 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.210585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.224732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:53.230730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:53.238235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:53.239274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 46/90] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:53.241574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:53.356487 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc0d0d-6055-485c-b14b-da96dbde0d82 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:53.367666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 45/91] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:53.372201 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8e4bb-d441-45e3-8897-34c05a4d5bb2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.372551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8e4bb-d441-45e3-8897-34c05a4d5bb2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.372843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8e4bb-d441-45e3-8897-34c05a4d5bb2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.374646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 46/92] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:53.380956 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.381254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.397847 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:53.404559 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:53.411424 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:53.413445 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:53.414449 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.414954 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.415482 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.416503 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] RBAC: Adding query filter params (name=member) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:53.417032 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:53.423136 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:53.434160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 47/93] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:39:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:53.448907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:53.537641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 47/94] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:53.619089 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1edf8-2819-490d-8fdc-ac3814b24386 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:53.663739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:53.664715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.665239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.665753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] PATH_INFO: `/v3/users/dc60bb94695043b38502ba313cbe55ad` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.668073 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] RBAC: Authorizing `identity:get_user(user_id=dc60bb94695043b38502ba313cbe55ad)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:53.669791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:53.671964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 48/95] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:53 2022] GET /identity/v3/users/dc60bb94695043b38502ba313cbe55ad => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:39:53.680415 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:53.716204 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:53.802222 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:53.831687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:53.875693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0467398-11c8-4be5-aab2-53f79eed6fcb None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:53.946495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:53.947174 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:53.947492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:53.947774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:53.953508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3707f-3016-4dfc-a34e-ad6e6efce003 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.037530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.038186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.038476 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.038759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.061143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:54.066175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:54.067780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:54.068729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 49/96] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:39:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:39:54.077794 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.088782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.090282 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.162455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:54.221346 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 48/97] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:39:54.222652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.224300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa809192-6583-4305-a9e8-29fbf4fc4f09 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.246053 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:54.268300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692aa8c-52a0-49c6-b551-54becad103c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.268960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692aa8c-52a0-49c6-b551-54becad103c8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.269253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692aa8c-52a0-49c6-b551-54becad103c8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.270071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692aa8c-52a0-49c6-b551-54becad103c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.283746 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692aa8c-52a0-49c6-b551-54becad103c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d692aa8c-52a0-49c6-b551-54becad103c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:54.290258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:54.291285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:54.292514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 50/98] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:54.296795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402e8ce-14c5-4dbb-9023-1c5712671027 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.298042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402e8ce-14c5-4dbb-9023-1c5712671027 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.299160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402e8ce-14c5-4dbb-9023-1c5712671027 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.301904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 51/99] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:54.302473 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402e8ce-14c5-4dbb-9023-1c5712671027 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.359717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f28-a183-4e9b-bff4-a1a3f1b883db None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.412582 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.414684 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.415519 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.416035 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.417264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:54.417820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.420485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.430182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402e8ce-14c5-4dbb-9023-1c5712671027 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.434850 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 49/100] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:54.436103 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.456964 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:54.467844 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a6774-2294-4049-86d8-f375d61fcd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.469046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a6774-2294-4049-86d8-f375d61fcd1b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.469046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a6774-2294-4049-86d8-f375d61fcd1b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.469252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a6774-2294-4049-86d8-f375d61fcd1b None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.470301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a6774-2294-4049-86d8-f375d61fcd1b None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.471893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a6774-2294-4049-86d8-f375d61fcd1b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.474079 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 52/101] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:54.475205 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.475528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.475850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.494961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:54.496209 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:54.537552 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:54.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ee0e2-21b6-42f9-bb4b-660f24bf02ce None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.564663 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 53/102] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:54.575446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.598436 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.599165 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.599484 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.599768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.600442 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:54.600757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.602149 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6095d-8160-4000-9c4c-a41d217ef631 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.612196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 50/103] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:54.613778 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f2f04-e9ae-44b8-bc36-e6939ca22db1 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.614083 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f2f04-e9ae-44b8-bc36-e6939ca22db1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.614358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f2f04-e9ae-44b8-bc36-e6939ca22db1 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.615598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 51/104] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:54.620651 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f2f04-e9ae-44b8-bc36-e6939ca22db1 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.666799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2e985-981f-445a-8769-70e27b13aa42 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.705441 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbde43-ff11-4735-8479-2cd9d19d8bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.706077 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbde43-ff11-4735-8479-2cd9d19d8bba None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.706372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbde43-ff11-4735-8479-2cd9d19d8bba None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.706662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbde43-ff11-4735-8479-2cd9d19d8bba None admin] PATH_INFO: `/v3/users/dc60bb94695043b38502ba313cbe55ad` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.708000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbde43-ff11-4735-8479-2cd9d19d8bba None admin] RBAC: Authorizing `identity:get_user(user_id=dc60bb94695043b38502ba313cbe55ad)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.709376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:54.709496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbde43-ff11-4735-8479-2cd9d19d8bba None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.711216 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 54/105] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/users/dc60bb94695043b38502ba313cbe55ad => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:54.712273 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211a46-88cb-412d-b6a7-e570c317c1a0 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.712573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211a46-88cb-412d-b6a7-e570c317c1a0 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.712855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211a46-88cb-412d-b6a7-e570c317c1a0 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.714212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 55/106] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:54.718960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211a46-88cb-412d-b6a7-e570c317c1a0 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.740966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f2f04-e9ae-44b8-bc36-e6939ca22db1 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.767881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:54.776437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.778004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.778004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.778004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.778377 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] RBAC: Adding query filter params (role.id=153c5cb975cc4d2b868b1cf6ce5331cc, scope.project.id=8a1cdee355bb441ea1a53da3a457e54e, user.id=dc60bb94695043b38502ba313cbe55ad) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:54.778616 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.780094 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f21642-b020-454b-bc2c-45f72e65a9be None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.789087 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 52/107] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/role_assignments?user.id=dc60bb94695043b38502ba313cbe55ad&scope.project.id=8a1cdee355bb441ea1a53da3a457e54e&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:54.792114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.792114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.792114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.815787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:54.817295 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:54.862212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211a46-88cb-412d-b6a7-e570c317c1a0 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:54.897492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 53/108] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:54.898199 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:54.907362 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8789a-5a81-4eba-87f1-c30d8723331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:54.908405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8789a-5a81-4eba-87f1-c30d8723331e None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.909349 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8789a-5a81-4eba-87f1-c30d8723331e None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.909876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8789a-5a81-4eba-87f1-c30d8723331e None admin] PATH_INFO: `/v3/users/437c2d620e4648bb9b558b4098744fa0` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.912146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8789a-5a81-4eba-87f1-c30d8723331e None admin] RBAC: Authorizing `identity:get_user(user_id=437c2d620e4648bb9b558b4098744fa0)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:54.914558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8789a-5a81-4eba-87f1-c30d8723331e None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:54.917050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 56/109] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/users/437c2d620e4648bb9b558b4098744fa0 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:54.918916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:54.919464 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:54.919938 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:54.946821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:54.948710 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:54.975682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.019660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c15f3f-32d2-4750-a5c6-02b03b1d90cd None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.035559 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 57/110] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:55.044203 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.060447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.061074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.061562 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.061678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.103265 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.140846 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.142785 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.183094 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7f3f3-838d-4863-a691-bf679af01bbf None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.215934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.224503 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.230036 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.230036 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.230036 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.236297 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:55.241323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:55.242558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:39:55.243880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 58/111] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:55.248109 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.260954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 54/112] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:39:55.260954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.349731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.352808 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459369-2209-4c26-ac2f-0aa9f9826c44 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.362766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03bd6-a23a-4813-847e-6d79eaeef8b8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.387797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.388408 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.388696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.388984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.397022 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.397654 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.397943 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.398224 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.409085 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:55.414493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:55.416031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 55/113] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:39:55.416987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.446111 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.456649 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.458175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.517971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab159ad9-99a9-4e5d-acee-e6d5d0b93d18 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.552388 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.552553 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.553002 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.553289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.553572 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.554238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:55.554549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.555983 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.559991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 59/114] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:39:55.560216 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.564597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 56/115] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:55.564597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.658097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564d1b-9a82-4776-b17c-ce380fe400d3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.659638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.669584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dabb2-3053-4c2d-ab70-e8babd18b5ea None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.696037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.696727 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.696961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.697246 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.698386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.699875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.701529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 60/116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:55.702288 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.706506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.707693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.707988 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.708281 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.709423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.710893 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.715716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 57/117] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:55.715716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.773000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.808708 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92571d-6a21-4014-99c1-f4173c37255e None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.814672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0486-fffe-4eee-bfee-525b9026f13d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.843136 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.843778 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.844071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.844350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.845018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:55.845328 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.846710 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.849413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.850046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.850338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.850623 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.851785 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.853210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.854865 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 58/119] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:55.855669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.859531 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 61/119] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:55.860432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:55.884731 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.959104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e5d8a-c6f5-4ed7-b086-c56ca35dede3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.959496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf0b46-7656-4515-8062-41d20be657ac None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:55.987452 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:55.993010 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.993190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:55.993652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.993773 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:55.993942 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.994046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:55.994230 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] PATH_INFO: `/v3/users/dc60bb94695043b38502ba313cbe55ad` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.994346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:55.994973 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] RBAC: Adding query filter params (role.id=c040847a15b148f893d961a9a578db29, scope.project.id=8a1cdee355bb441ea1a53da3a457e54e, user.id=dc60bb94695043b38502ba313cbe55ad) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:55.995290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.995427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=dc60bb94695043b38502ba313cbe55ad)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:55.996765 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.996863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:55.998572 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 59/120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/users/dc60bb94695043b38502ba313cbe55ad => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:55.999154 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.018857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 62/121] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:39:55 2022] GET /identity/v3/role_assignments?user.id=dc60bb94695043b38502ba313cbe55ad&scope.project.id=8a1cdee355bb441ea1a53da3a457e54e&role.id=c040847a15b148f893d961a9a578db29 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:56.018857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.119703 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:56.120380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd70e59-4760-4ccb-96e2-bd9728af3422 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.126556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78587a99-5a59-4af4-8316-c94836ca1ec3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.158874 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.159851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.160822 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.161105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] PATH_INFO: `/v3/users/dab74e443ca9493c90bb1fe4370f8685` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.165390 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.166033 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.166327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.166620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.167329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.168835 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.176668 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] RBAC: Authorizing `identity:get_user(user_id=dab74e443ca9493c90bb1fe4370f8685)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.178157 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.179894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 63/122] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:56 2022] GET /identity/v3/users/dab74e443ca9493c90bb1fe4370f8685 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:56.180674 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.239481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:56.250639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 60/123] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:39:55 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:56.251564 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.290215 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8213d2-6720-42f9-90b1-4f8f4256a325 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.327930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.328568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.328861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.329149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.329921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=8a1cdee355bb441ea1a53da3a457e54e, user.id=437c2d620e4648bb9b558b4098744fa0) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:56.330248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.331799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.338340 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 64/124] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:39:56 2022] GET /identity/v3/role_assignments?user.id=437c2d620e4648bb9b558b4098744fa0&scope.project.id=8a1cdee355bb441ea1a53da3a457e54e&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:56.339229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.383582 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4490a-189c-4edc-b8c7-a9032c3e249a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.437347 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.438153 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.438545 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.438930 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.464485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:56.489882 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66ee79-922f-4633-8541-e1c8311087a1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.516156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:56.532031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.533988 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.536221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.537071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.537362 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.538537 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.540111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.541976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.545054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:56 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:56.550276 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.669386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31445d5-165f-4b68-a4c3-5e5248da7c90 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.677705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 61/126] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:39:56.683515 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.704320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.704993 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.705286 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.705570 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/dab74e443ca9493c90bb1fe4370f8685/roles/77152c82ebb94862b2211a7783423231` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.707565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a1cdee355bb441ea1a53da3a457e54e, user_id=dab74e443ca9493c90bb1fe4370f8685, role_id=77152c82ebb94862b2211a7783423231)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.709010 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.731344 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:56.768859 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 66/127] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:39:56 2022] PUT /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/users/dab74e443ca9493c90bb1fe4370f8685/roles/77152c82ebb94862b2211a7783423231 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:39:56.786787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae039f9-da4a-4462-864a-806b7e39a284 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.833997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95d021-c577-48ea-b530-93f2c7b5c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.834669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d021-c577-48ea-b530-93f2c7b5c850 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.834957 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d021-c577-48ea-b530-93f2c7b5c850 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.835241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d021-c577-48ea-b530-93f2c7b5c850 None admin] PATH_INFO: `/v3/projects/e2762dd6d69f4823859cf55d19460c63` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.836464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95d021-c577-48ea-b530-93f2c7b5c850 None admin] RBAC: Authorizing `identity:get_project(project_id=e2762dd6d69f4823859cf55d19460c63)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.837973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95d021-c577-48ea-b530-93f2c7b5c850 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:56.839922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 62/128] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:39:56 2022] GET /identity/v3/projects/e2762dd6d69f4823859cf55d19460c63 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:56.846045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:56.897691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:56.957688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de1b30-78b9-422c-8de9-b9b66f87dfd1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:56.993225 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:56.993852 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:56.994141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:56.994426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:56.995160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] RBAC: Adding query filter params (role.id=153c5cb975cc4d2b868b1cf6ce5331cc, scope.project.id=e2762dd6d69f4823859cf55d19460c63, user.id=dc60bb94695043b38502ba313cbe55ad) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:56.995511 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:56.996962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74498f3e-9dc5-42c4-803e-dfa64ecd91d7 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:57.003052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 67/129] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:39:56 2022] GET /identity/v3/role_assignments?user.id=dc60bb94695043b38502ba313cbe55ad&scope.project.id=e2762dd6d69f4823859cf55d19460c63&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:58.526092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf68a-ae45-4c97-a37d-507f6d3926a1 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:58.526641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf68a-ae45-4c97-a37d-507f6d3926a1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:58.526641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf68a-ae45-4c97-a37d-507f6d3926a1 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:58.528059 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 63/130] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:39:58.533470 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31cd94-fa0b-4324-8917-99514e2b377f None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:58.533900 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31cd94-fa0b-4324-8917-99514e2b377f None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:58.534046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31cd94-fa0b-4324-8917-99514e2b377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:58.553582 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-6b31cd94-fa0b-4324-8917-99514e2b377f None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:58.554784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-6b31cd94-fa0b-4324-8917-99514e2b377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:58.621256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 68/131] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:39:58.889341 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf68a-ae45-4c97-a37d-507f6d3926a1 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:58.901531 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a359d-6610-4b38-8e10-09d37e7dc3cb None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:58.901820 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a359d-6610-4b38-8e10-09d37e7dc3cb None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:58.902110 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a359d-6610-4b38-8e10-09d37e7dc3cb None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:58.903500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 69/132] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:39:58.908338 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:58.908628 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:58.908920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:58.927443 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:39:58.928654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:39:58.958187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:58.996832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 70/133] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:39:58.997820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf68a-ae45-4c97-a37d-507f6d3926a1 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:59.015137 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:59.038056 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:59.038751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:59.039101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:59.039443 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:59.040675 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:59.042120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:59.045513 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 64/134] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:59.049894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:59.150707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f313d-9403-4cc2-bf63-6c8d753e0405 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:59.184577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:59.185185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:59.185469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:59.185748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:59.186397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:59.187878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:59.264220 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:59.271858 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 71/135] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:39:59 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:39:59.372066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59531a-f31e-4a34-a754-aec364e7c0d4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:59.407519 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:59.408184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:59.408476 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:59.408761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:59.420551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:39:59.426371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:39:59.427597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:39:59.428578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 65/136] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:39:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:39:59.433542 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:59.461045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:59.539974 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82b26e-47a9-4a78-be6c-0363a8c4b6dc None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:59.576372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:59.577003 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:59.577292 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:59.577572 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:59.578251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] RBAC: Adding query filter params (name=reader) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:39:59.578563 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:59.579985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:59.589245 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 72/137] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:39:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:59.594709 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:59.643618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:59.709285 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b985e60-c6a6-4124-af43-3de0ea27ccef None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:59.746484 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:59.747626 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:59.748145 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:59.748648 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] PATH_INFO: `/v3/users/dab74e443ca9493c90bb1fe4370f8685` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:59.750064 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] RBAC: Authorizing `identity:get_user(user_id=dab74e443ca9493c90bb1fe4370f8685)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:59.751574 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:39:59.753353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 66/138] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:39:59 2022] GET /identity/v3/users/dab74e443ca9493c90bb1fe4370f8685 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:39:59.758789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:39:59.856312 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:39:59.872081 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aae5e-6b09-482b-90b8-f3db7da4995a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:39:59.908608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:39:59.909597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:39:59.910624 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:39:59.911123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:39:59.983613 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:39:59.985483 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:00.074324 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:00.091451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 73/139] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:39:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:00.100473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:00.208252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37958636-353e-44ec-80c6-aa1da5ec374f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:00.222934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:00.247972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76487e8-0f86-46ab-b7a0-6663ca4d5b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:00.248689 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76487e8-0f86-46ab-b7a0-6663ca4d5b52 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:00.248976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76487e8-0f86-46ab-b7a0-6663ca4d5b52 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:00.249259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76487e8-0f86-46ab-b7a0-6663ca4d5b52 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:00.250413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76487e8-0f86-46ab-b7a0-6663ca4d5b52 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:00.251914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76487e8-0f86-46ab-b7a0-6663ca4d5b52 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:00.254745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 67/140] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:00 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:00.260653 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:00.365864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff3db5-00f5-466f-af58-ae6bba3ab6d6 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:00.401348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:00.401968 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:00.402254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:00.402536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:00.403285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] RBAC: Adding query filter params (role.id=77152c82ebb94862b2211a7783423231, scope.project.id=8a1cdee355bb441ea1a53da3a457e54e, user.id=dab74e443ca9493c90bb1fe4370f8685) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:00.403707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:00.405257 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528fc6-d619-44ca-b802-7198990b09a9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:00.412499 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 74/141] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:00 2022] GET /identity/v3/role_assignments?user.id=dab74e443ca9493c90bb1fe4370f8685&scope.project.id=8a1cdee355bb441ea1a53da3a457e54e&role.id=77152c82ebb94862b2211a7783423231 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:01.561228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19d9c-a695-4f9a-82b9-55a9eb755d6b None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:01.561804 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19d9c-a695-4f9a-82b9-55a9eb755d6b None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:01.561804 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19d9c-a695-4f9a-82b9-55a9eb755d6b None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:01.563136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 68/142] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:01.568874 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5040a8-e80d-4da8-96f6-9896e26a20ee None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:01.569323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5040a8-e80d-4da8-96f6-9896e26a20ee None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:01.569420 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5040a8-e80d-4da8-96f6-9896e26a20ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:01.589228 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-2b5040a8-e80d-4da8-96f6-9896e26a20ee None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:01.590462 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-2b5040a8-e80d-4da8-96f6-9896e26a20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:01.641920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:01.662214 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 75/143] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:01.708037 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762cc3-f8b9-4137-95f5-2fb5a4113a68 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:01.708336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762cc3-f8b9-4137-95f5-2fb5a4113a68 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:01.708763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4762cc3-f8b9-4137-95f5-2fb5a4113a68 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:01.710338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 69/144] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:01.715567 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c0328-9f25-4d82-b492-0da98ab78cf6 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:01.715866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c0328-9f25-4d82-b492-0da98ab78cf6 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:01.716151 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c0328-9f25-4d82-b492-0da98ab78cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:01.734955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-bc4c0328-9f25-4d82-b492-0da98ab78cf6 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:01.736187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-bc4c0328-9f25-4d82-b492-0da98ab78cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:01.815003 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 76/145] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:01.870856 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9c9c-da0e-40f0-b05e-ad5955e36877 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:01.871180 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9c9c-da0e-40f0-b05e-ad5955e36877 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:01.871485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9c9c-da0e-40f0-b05e-ad5955e36877 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:01.872973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 70/146] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:01.878142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054180-9134-4520-b4b0-688b86e16715 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:01.878451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054180-9134-4520-b4b0-688b86e16715 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:01.878771 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054180-9134-4520-b4b0-688b86e16715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:01.897641 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-47054180-9134-4520-b4b0-688b86e16715 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:01.898849 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-47054180-9134-4520-b4b0-688b86e16715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:01.929258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c9c9c-da0e-40f0-b05e-ad5955e36877 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:01.971008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 77/147] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:02.036114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c9c9c-da0e-40f0-b05e-ad5955e36877 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.069919 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.070506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.070802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.071089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.081409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054180-9134-4520-b4b0-688b86e16715 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.081581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:02.087535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:02.090535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:02.091529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 71/148] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:02.094410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.127826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.138941 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.189861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054180-9134-4520-b4b0-688b86e16715 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.201480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acfb36-bb57-44f8-8b75-32af027c0529 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.225709 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.226351 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.226665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.226917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.237894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:02.240146 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.240766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.241063 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.241347 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.242023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:02.242338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:02.242898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:02.244107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:02.245040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:02.245765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.245840 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 78/149] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:02.245840 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.255327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 72/150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:02.259815 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.349691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.352660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849f84-eac0-4aa3-a93a-fc7562e6848a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.355003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.357738 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e575e4-fd3b-4b3d-9bd7-6202f98d4cf3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.387888 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.388516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.388822 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.389124 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.390256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.391777 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.393367 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 79/151] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:02.393820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.394168 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.394507 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.394755 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.395041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.395751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:02.396066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.397451 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.407517 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 73/152] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:02.408304 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.496199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128c1-e1c4-4da8-a6d8-7dbb36e7e465 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.509408 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e934deb-ddec-4a3a-afcf-e87c57a7a642 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.530033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.530668 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.530956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.531239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] PATH_INFO: `/v3/users/a45ba456c912480ebe62dedef2d3c737` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.542353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.542960 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.543249 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.543569 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.544222 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.544331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] RBAC: Authorizing `identity:get_user(user_id=a45ba456c912480ebe62dedef2d3c737)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.545668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.545794 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.547629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 80/153] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/users/a45ba456c912480ebe62dedef2d3c737 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:02.548404 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.576919 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.625503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 74/154] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:02 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:02.626277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.649673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.652959 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3cb4c-6c1d-45ac-9e8c-63777ddbc30b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.692051 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.692719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.693008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.693283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] PATH_INFO: `/v3/users/a45ba456c912480ebe62dedef2d3c737` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.694483 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] RBAC: Authorizing `identity:get_user(user_id=a45ba456c912480ebe62dedef2d3c737)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.696001 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.697844 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 81/155] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/users/a45ba456c912480ebe62dedef2d3c737 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:02.703547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.752657 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3b748-4fab-43c4-8b3e-ea0cf00b45da None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.792571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.793626 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.794124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.794608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.796603 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.798138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.800784 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 75/156] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:02.805813 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.826021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.834750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650dea6-f603-417e-8934-b5c9fdfe2659 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.875235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.878403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.878403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.878403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.878403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.879640 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:02.881982 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 82/157] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:02 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:02.888185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:02.914361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:02.936552 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93077-5d10-4d43-88ef-7479d21fdd5d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:02.982119 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94620dce-222b-4bbd-a859-c809ff78b7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:02.983181 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94620dce-222b-4bbd-a859-c809ff78b7b4 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:02.983762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94620dce-222b-4bbd-a859-c809ff78b7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:02.984292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94620dce-222b-4bbd-a859-c809ff78b7b4 None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/a45ba456c912480ebe62dedef2d3c737/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:02.989071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94620dce-222b-4bbd-a859-c809ff78b7b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ad5db5d05144b98ec29f5c46a876a7, user_id=a45ba456c912480ebe62dedef2d3c737, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:02.989071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94620dce-222b-4bbd-a859-c809ff78b7b4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:03.014279 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826635c-fca9-48c6-b91a-4fe41fb3c9a0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:03.050814 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7856-0bcd-4f3c-ac46-edadb6608a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:03.051514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7856-0bcd-4f3c-ac46-edadb6608a3c None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:03.051800 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7856-0bcd-4f3c-ac46-edadb6608a3c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:03.052076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7856-0bcd-4f3c-ac46-edadb6608a3c None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/a45ba456c912480ebe62dedef2d3c737/roles/c040847a15b148f893d961a9a578db29` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:03.054437 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7856-0bcd-4f3c-ac46-edadb6608a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=73ad5db5d05144b98ec29f5c46a876a7, user_id=a45ba456c912480ebe62dedef2d3c737, role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:03.055951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7856-0bcd-4f3c-ac46-edadb6608a3c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:03.078537 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 76/158] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:02 2022] PUT /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/a45ba456c912480ebe62dedef2d3c737/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:40:03.140642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 83/159] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:02 2022] PUT /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/a45ba456c912480ebe62dedef2d3c737/roles/c040847a15b148f893d961a9a578db29 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:40:04.888698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df69f8-7fed-4222-b03f-e131b16d7c5d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:04.889251 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df69f8-7fed-4222-b03f-e131b16d7c5d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:04.889251 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df69f8-7fed-4222-b03f-e131b16d7c5d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:04.891018 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 77/160] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:04.897771 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e10a-ef70-47e5-bc6a-6c96975f9126 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:04.898310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e10a-ef70-47e5-bc6a-6c96975f9126 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:04.898310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e10a-ef70-47e5-bc6a-6c96975f9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:04.917749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-f7b5e10a-ef70-47e5-bc6a-6c96975f9126 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:04.918945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-f7b5e10a-ef70-47e5-bc6a-6c96975f9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:04.936868 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:04.995419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 84/161] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:05.000736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9bd7-df73-4cf2-8720-728ebf8da7bf None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.001030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9bd7-df73-4cf2-8720-728ebf8da7bf None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.001315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9bd7-df73-4cf2-8720-728ebf8da7bf None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.002680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 78/162] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:05.009048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2ee3b-57c5-4d83-9740-1aa8de6ab285 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.009521 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2ee3b-57c5-4d83-9740-1aa8de6ab285 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.010239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2ee3b-57c5-4d83-9740-1aa8de6ab285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.028590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-cab2ee3b-57c5-4d83-9740-1aa8de6ab285 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:05.029777 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-cab2ee3b-57c5-4d83-9740-1aa8de6ab285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:05.104732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 85/163] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:05.272528 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9bd7-df73-4cf2-8720-728ebf8da7bf None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.282177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcd0b2-17da-4b03-ba72-e8e3cb2af460 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.282464 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcd0b2-17da-4b03-ba72-e8e3cb2af460 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.282754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcd0b2-17da-4b03-ba72-e8e3cb2af460 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.284398 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 86/164] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:05.290755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.291096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.291367 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.311017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:05.312516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:05.385545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 87/165] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:05.392274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.403363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9bd7-df73-4cf2-8720-728ebf8da7bf None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:05.410423 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:05.441661 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d455990-9272-4aab-928f-061714ed0fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:05.442532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d455990-9272-4aab-928f-061714ed0fd3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.442812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d455990-9272-4aab-928f-061714ed0fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.443081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d455990-9272-4aab-928f-061714ed0fd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.453565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d455990-9272-4aab-928f-061714ed0fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-3d455990-9272-4aab-928f-061714ed0fd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:05.458470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:05.459742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:05.460999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 79/166] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:05.461783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66cec6-1ce8-45a6-affa-f58d3139bc09 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.461783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66cec6-1ce8-45a6-affa-f58d3139bc09 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.461783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66cec6-1ce8-45a6-affa-f58d3139bc09 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.462948 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 80/167] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:05.465218 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66cec6-1ce8-45a6-affa-f58d3139bc09 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.474276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:05.515326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f86fe-96ef-4e6f-8937-304d1d3e767c None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:05.549085 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef14474-32aa-48e7-85e4-9dcfa4693b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:05.549718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef14474-32aa-48e7-85e4-9dcfa4693b82 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.550003 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef14474-32aa-48e7-85e4-9dcfa4693b82 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.550802 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef14474-32aa-48e7-85e4-9dcfa4693b82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.551928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef14474-32aa-48e7-85e4-9dcfa4693b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:05.553375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef14474-32aa-48e7-85e4-9dcfa4693b82 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:05.554954 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 88/168] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:05.556157 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.556446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.556724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.575168 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:05.576392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:05.586845 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66cec6-1ce8-45a6-affa-f58d3139bc09 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:05.631932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:05.632871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.633357 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.634338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.635499 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:05.636063 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:05.637827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:05.647183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 89/169] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:05.647349 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 81/170] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:05.648126 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.659109 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.670816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:05.725281 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:05.763599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370037dd-3925-4fa8-9741-4e67f6a0f6e1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:05.764345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556004a3-a054-44e6-97d6-c1c714c26cce None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:05.799255 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:05.802175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.802175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.802175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.802175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:05.802717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:05.806372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:05.807054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.807382 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.807670 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] PATH_INFO: `/v3/users/7b16cd145e2c46e49c36d20e2c2bbe54` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.822053 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] RBAC: Authorizing `identity:get_user(user_id=7b16cd145e2c46e49c36d20e2c2bbe54)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:05.823532 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:05.825169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 90/171] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:05 2022] GET /identity/v3/users/7b16cd145e2c46e49c36d20e2c2bbe54 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:05.825936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.877460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 82/172] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:05 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:05.878770 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:05.919812 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:05.943286 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18828c4-b1a6-4265-90de-a0de0c742915 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:05.983015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:05.984054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:05.984054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:05.984365 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:05.996600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:06.001673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:06.002739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 91/173] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:06.004350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.019341 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5f75-1348-40fa-98b1-52e3d3c5b688 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.045644 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.058545 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.059350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.060308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.060308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.061123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.062633 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.064371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 83/174] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:05 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:06.068777 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.127591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0550f5-0dc0-4aba-a5c5-d918ed1f6ce2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.165538 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.181832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.182409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.182692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.182967 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.189457 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c5628-1bfa-47c4-ac1d-612605d33295 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.196692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:06.201933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:06.202991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:06.204029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 92/175] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:06.204821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.235493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.236300 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.236690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.237072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.237795 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:06.238111 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.239542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.248521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 84/176] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:06.249893 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.318332 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.324428 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b070f7-2e6a-4c22-b8ed-ced4c7837c7f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.372338 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.372727 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b525dc-9f43-4c0f-b7ce-b289ad2769df None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.373610 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.374363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.375607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/7b16cd145e2c46e49c36d20e2c2bbe54/roles/153c5cb975cc4d2b868b1cf6ce5331cc` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.377597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] RBAC: Authorizing `identity:create_grant(project_id=73ad5db5d05144b98ec29f5c46a876a7, user_id=7b16cd145e2c46e49c36d20e2c2bbe54, role_id=153c5cb975cc4d2b868b1cf6ce5331cc)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.379912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.410423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.411045 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.411360 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.411645 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.412317 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:06.412635 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.414053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.422902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 85/177] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:06.428705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.452267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 93/178] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:06 2022] PUT /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/7b16cd145e2c46e49c36d20e2c2bbe54/roles/153c5cb975cc4d2b868b1cf6ce5331cc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:06.454274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.535747 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.550292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2440c0-5001-43b7-a4fc-550a6209e4cd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.574806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.591119 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01ece6-c03b-4b5e-9cc2-a64c8e105faf None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.606452 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.606704 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.609414 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.609642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] PATH_INFO: `/v3/users/a45ba456c912480ebe62dedef2d3c737` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.612147 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] RBAC: Authorizing `identity:get_user(user_id=a45ba456c912480ebe62dedef2d3c737)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.614532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.618308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 86/179] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/users/a45ba456c912480ebe62dedef2d3c737 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:06.626056 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.649699 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.651789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.651789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.651789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] PATH_INFO: `/v3/users/a45ba456c912480ebe62dedef2d3c737` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.653051 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] RBAC: Authorizing `identity:get_user(user_id=a45ba456c912480ebe62dedef2d3c737)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.655496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.658224 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 94/180] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/users/a45ba456c912480ebe62dedef2d3c737 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:06.667728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:06.799247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45683-5aa4-4f0d-907f-fc85e1cf9736 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.822502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.839387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9880b-9523-4b48-bac2-c505093bc2aa None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:06.842623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.843523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.847056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.847056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.885118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:06.885847 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:06.886205 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:06.886563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:06.950647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:06.952141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.953393 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:06.971160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:06.973625 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:07.096670 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 87/181] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:07.104477 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:07.182593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 95/182] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:07.184923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:07.213859 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:07.249523 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:07.279143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1155a46-c32b-482f-aa69-f229bc99a9de None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:07.357076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:07.363352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:07.363352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:07.363352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:07.363352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:07.363352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:07.363352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 88/183] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:07 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:07.369041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:07.383736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353abec-3ca5-4276-b5cf-b9336ab06699 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:07.450260 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:07.451163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:07.451642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:07.452089 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:07.454141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:07.455793 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:07.464265 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 96/184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:07 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:07.469946 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:07.554615 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78d9ed-72ab-4958-a423-337d08846d3d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:07.621740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:07.622702 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:07.623156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:07.623609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:07.624515 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=73ad5db5d05144b98ec29f5c46a876a7, user.id=a45ba456c912480ebe62dedef2d3c737) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:07.625002 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:07.627573 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2d2e-455e-4d1e-b05a-d918292f719c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:07.639589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 89/185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:07 2022] GET /identity/v3/role_assignments?user.id=a45ba456c912480ebe62dedef2d3c737&scope.project.id=73ad5db5d05144b98ec29f5c46a876a7&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:07.647694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:07.655862 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6668f41-0705-4ef2-9019-4b081851c68b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:07.694331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:07.695453 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:07.695957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:07.696426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:07.697393 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] RBAC: Adding query filter params (role.id=c040847a15b148f893d961a9a578db29, scope.project.id=73ad5db5d05144b98ec29f5c46a876a7, user.id=a45ba456c912480ebe62dedef2d3c737) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:07.697975 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:07.699710 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220be0f-8391-41d3-987c-8162bcdc948d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:07.708169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 97/186] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:07 2022] GET /identity/v3/role_assignments?user.id=a45ba456c912480ebe62dedef2d3c737&scope.project.id=73ad5db5d05144b98ec29f5c46a876a7&role.id=c040847a15b148f893d961a9a578db29 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:08.392331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937651-c7bb-422a-ad34-e001fabf7487 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:08.392925 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937651-c7bb-422a-ad34-e001fabf7487 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:08.392925 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937651-c7bb-422a-ad34-e001fabf7487 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:08.394914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 90/187] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:08.397384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e45e-ff2f-44db-9e0e-a7a4efb46a2a None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:08.397808 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e45e-ff2f-44db-9e0e-a7a4efb46a2a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:08.397984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e45e-ff2f-44db-9e0e-a7a4efb46a2a None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:08.399728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 98/188] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:08.400051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:08.400347 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:08.400632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:08.404966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:08.405260 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:08.405544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:08.419829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:08.421025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:08.424254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:08.425439 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:08.433496 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:08.492934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 91/189] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:08.495419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 99/190] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:08.761974 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:08.769206 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:08.857785 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:08.879189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558294e-0069-47d3-aaf4-78c585b097b4 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:08.879798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74951b-5759-4aab-8e60-dd5cc25b9be0 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:08.916176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:08.916547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:08.916819 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:08.917101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:08.917207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:08.917383 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:08.917485 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:08.917692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:08.918790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:08.920296 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:08.922300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 100/191] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:08.928113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:08.929305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:08.934315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:08.935792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:08.936734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:08.940660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 92/192] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:08.941903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:08.952102 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:09.035078 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5ce82-12cd-42c3-a35c-b203ec75e5a3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.057221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e10af8-a3a4-4a2d-af35-5e9948f79b1e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.069913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c171ddc-7b30-4959-b3ea-c17b1ee68895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.070527 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c171ddc-7b30-4959-b3ea-c17b1ee68895 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.070807 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c171ddc-7b30-4959-b3ea-c17b1ee68895 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.071080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c171ddc-7b30-4959-b3ea-c17b1ee68895 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.071758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c171ddc-7b30-4959-b3ea-c17b1ee68895 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:09.073409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c171ddc-7b30-4959-b3ea-c17b1ee68895 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:09.093279 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.093892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.094179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.094459 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.095127 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:09.095513 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:09.096887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442000ed-45a3-49b4-917f-326e20accdd7 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:09.105543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 93/193] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:09.106714 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c2bd2-7374-4aef-9ac9-b7586c687e13 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.107004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c2bd2-7374-4aef-9ac9-b7586c687e13 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.112810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c2bd2-7374-4aef-9ac9-b7586c687e13 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.112810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 94/194] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:09.112810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c2bd2-7374-4aef-9ac9-b7586c687e13 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:09.146046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 101/195] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:08 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:40:09.147537 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.147824 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.148101 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.163102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:09.176568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:09.177896 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:09.186144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:09.229876 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c2bd2-7374-4aef-9ac9-b7586c687e13 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.264070 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 102/196] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:09.267416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.268157 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.268514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.269375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] PATH_INFO: `/v3/users/73444d7f4ea44039aabe4dad444f1788` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.284968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] RBAC: Authorizing `identity:get_user(user_id=73444d7f4ea44039aabe4dad444f1788)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:09.286632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:09.288617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 95/197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:09 2022] GET /identity/v3/users/73444d7f4ea44039aabe4dad444f1788 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:09.294373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:09.400026 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa2855-7d9b-4312-af63-e60ede797d32 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.434131 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:09.441773 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.447735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.447735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.447735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.447735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:09.447735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:09.447735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 103/198] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:09 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:09.456672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:09.529916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:09.574634 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606f1cb-6546-4673-8851-c24b25cdacf6 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.606807 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:09.618206 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.618895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.619186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.619550 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/73444d7f4ea44039aabe4dad444f1788/roles/77152c82ebb94862b2211a7783423231` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.622735 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] RBAC: Authorizing `identity:create_grant(project_id=73ad5db5d05144b98ec29f5c46a876a7, user_id=73444d7f4ea44039aabe4dad444f1788, role_id=77152c82ebb94862b2211a7783423231)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:09.624254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:09.653829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b6ac-15f8-4d6a-af8c-c6e20231e20f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.681440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 96/199] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:09 2022] PUT /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/users/73444d7f4ea44039aabe4dad444f1788/roles/77152c82ebb94862b2211a7783423231 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:09.702518 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.703246 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.703611 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.703896 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.717741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:09.723648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:09.724734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:09.725666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 104/200] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:09.731177 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:09.858345 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d880-e7ab-454c-b07d-e8b153ac98ba None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:09.899064 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:09.899992 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:09.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:09.900563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:09.901252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:09.901564 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:09.902958 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:09.912460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 97/201] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:09.918196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:09.984125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:10.028717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a4249-3274-4f66-a327-8f38663b137f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:10.065953 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:10.066608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:10.066891 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:10.067172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] PATH_INFO: `/v3/users/7b16cd145e2c46e49c36d20e2c2bbe54` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:10.068395 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] RBAC: Authorizing `identity:get_user(user_id=7b16cd145e2c46e49c36d20e2c2bbe54)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:10.069870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:10.071801 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 105/202] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:09 2022] GET /identity/v3/users/7b16cd145e2c46e49c36d20e2c2bbe54 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:10.077850 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:10.133559 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:10.190503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489e27-7c72-4b01-8546-04d217228d09 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:10.227488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:10.228136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:10.228424 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:10.228696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:10.302833 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:10.311137 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:10.362530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:10.418584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 98/203] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:10.424611 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:10.527902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71a00e-1772-41cc-a974-3597f1b06e8d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:10.556074 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:10.564201 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf31cf7-da7c-4993-899c-c0353624609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:10.564852 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31cf7-da7c-4993-899c-c0353624609d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:10.565140 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31cf7-da7c-4993-899c-c0353624609d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:10.565420 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31cf7-da7c-4993-899c-c0353624609d None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:10.567080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf31cf7-da7c-4993-899c-c0353624609d None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:10.568580 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf31cf7-da7c-4993-899c-c0353624609d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:10.570541 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 106/204] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:10 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:10.576915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:10.689024 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e97e54-9dc4-4560-ab16-e7d79e5fa646 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:10.725915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:10.726540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:10.726832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:10.727118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:10.728001 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] RBAC: Adding query filter params (role.id=153c5cb975cc4d2b868b1cf6ce5331cc, scope.project.id=73ad5db5d05144b98ec29f5c46a876a7, user.id=7b16cd145e2c46e49c36d20e2c2bbe54) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:10.728328 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:10.729837 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b39d4-a53e-4490-b2ac-a85029272861 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:10.735921 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 99/205] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:10 2022] GET /identity/v3/role_assignments?user.id=7b16cd145e2c46e49c36d20e2c2bbe54&scope.project.id=73ad5db5d05144b98ec29f5c46a876a7&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:11.532464 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68834f8d-8105-4ecb-9932-7e5fc1105d43 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:11.533005 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68834f8d-8105-4ecb-9932-7e5fc1105d43 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:11.533005 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68834f8d-8105-4ecb-9932-7e5fc1105d43 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:11.534715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 107/206] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:11.539917 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:11.540210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:11.540487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:11.559612 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:11.560822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:11.569277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:11.584394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428b827-02ce-4390-94ab-03b12ae50e94 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:11.589947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428b827-02ce-4390-94ab-03b12ae50e94 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:11.589947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428b827-02ce-4390-94ab-03b12ae50e94 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:11.589947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 108/207] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:11.591557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12818-0d79-48d4-927a-644ee755f56e None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:11.591869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12818-0d79-48d4-927a-644ee755f56e None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:11.592137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12818-0d79-48d4-927a-644ee755f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:11.610552 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-70e12818-0d79-48d4-927a-644ee755f56e None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:11.611782 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-70e12818-0d79-48d4-927a-644ee755f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:11.631465 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 100/208] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:11.680237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 109/209] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:11.898376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:11.938280 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dae63-159a-4c3a-9a64-8f5eb0b82a99 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:11.938613 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dae63-159a-4c3a-9a64-8f5eb0b82a99 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:11.938848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dae63-159a-4c3a-9a64-8f5eb0b82a99 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:11.940292 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 110/210] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:11.945660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:11.945945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:11.946221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:11.965212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:11.966381 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:12.000496 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a36a68-1aab-4fa6-90b1-c5fb8c8c2d8e None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.034412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 111/211] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:12.035386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.040002 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.040773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.041070 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.041349 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.052535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:12.057363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:12.058376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:12.059330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 101/212] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:12.064210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.067877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:12.102789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:12.148450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa60361-cd85-45ae-8c91-7a83f38cd24a None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.168134 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88d59-c34d-4608-8ddd-b9a506f228fd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.183461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.184112 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.184400 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.184679 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.185911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.187398 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.188985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 112/213] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:12.194367 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.205033 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.205656 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.205948 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.206734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.207598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:12.207914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.209305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.223217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 102/214] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:12.224216 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.289467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:12.301424 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0d3b-a9c9-45c3-bb85-4aa65509ad42 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.325189 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:12.330232 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb1b4c-86b2-43bb-86dc-2ed5af798d42 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.335796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.336402 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.336685 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.338864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.338864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.339124 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.364321 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.364942 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.365225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.365501 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] PATH_INFO: `/v3/users/2bdd1d0d8fd240cfae77d3d5f822124c` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.378606 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] RBAC: Authorizing `identity:get_user(user_id=2bdd1d0d8fd240cfae77d3d5f822124c)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.380107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.381730 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 103/215] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/users/2bdd1d0d8fd240cfae77d3d5f822124c => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:12.382504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.418285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 113/216] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:12 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:40:12.418285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.497788 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549d9c8-39ff-4e44-bbd4-2c7e64c0d4cc None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.525416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:12.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df70bf4-a543-4e8a-9b45-fd2d31706a11 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.538512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.539143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.539473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.539753 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.554826 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:12.562166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:12.563406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:12.564369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:12.565432 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 104/217] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:12.571751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.577208 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.577767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.578058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.578344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.579091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] RBAC: Adding query filter params (role.id=153c5cb975cc4d2b868b1cf6ce5331cc, user.id=2bdd1d0d8fd240cfae77d3d5f822124c, scope.system=all) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:12.579523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.580994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.586495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 114/218] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/role_assignments?user.id=2bdd1d0d8fd240cfae77d3d5f822124c&scope.system=all&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:12.695382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3a9b-0670-4b32-b8ba-09a0a1b187ae None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.736797 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.737438 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.737787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.738074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.738819 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:12.739138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.740737 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.750369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 105/219] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:12.756769 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.822126 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:12.886423 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fcc78-b3c9-44e4-a140-1b0592aa8a73 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:12.923766 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:12.924674 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:12.925160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:12.925619 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] PATH_INFO: `/v3/users/73444d7f4ea44039aabe4dad444f1788` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:12.926955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] RBAC: Authorizing `identity:get_user(user_id=73444d7f4ea44039aabe4dad444f1788)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:12.928673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:12.930842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 115/220] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/users/73444d7f4ea44039aabe4dad444f1788 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:12.936322 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:12.987145 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:13.045795 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2a378-e3da-46d2-ab0a-3d2e883fd6a5 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:13.083898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:13.084555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:13.084836 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:13.090060 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:13.162869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:13.164701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:13.212460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:13.270384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 106/221] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:13.280412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:13.389893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe0eac-7bb8-41f3-aa7b-e2444e1b4241 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:13.404284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:13.428605 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad60c74-2776-4020-809e-09d4bea81e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:13.429547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60c74-2776-4020-809e-09d4bea81e1c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:13.430048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60c74-2776-4020-809e-09d4bea81e1c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:13.430537 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60c74-2776-4020-809e-09d4bea81e1c None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:13.431923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad60c74-2776-4020-809e-09d4bea81e1c None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:13.433610 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad60c74-2776-4020-809e-09d4bea81e1c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:13.435543 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 116/222] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:13 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:13.441691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:13.552664 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f08259-6695-4e8c-82d5-d079c83c4a33 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:13.588407 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:13.589320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:13.589801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:13.590270 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:13.591194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] RBAC: Adding query filter params (role.id=77152c82ebb94862b2211a7783423231, scope.project.id=73ad5db5d05144b98ec29f5c46a876a7, user.id=73444d7f4ea44039aabe4dad444f1788) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:13.591727 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:13.593400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13164233-8313-4577-8e3d-ab3128f12f87 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:13.600243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 107/223] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:13 2022] GET /identity/v3/role_assignments?user.id=73444d7f4ea44039aabe4dad444f1788&scope.project.id=73ad5db5d05144b98ec29f5c46a876a7&role.id=77152c82ebb94862b2211a7783423231 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:14.795821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee409-c5ee-4bfb-ae7a-a59bbf7d9329 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:14.796599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a3cf-6d66-4d2e-ab7d-10e1f585cf5a None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:14.796957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee409-c5ee-4bfb-ae7a-a59bbf7d9329 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:14.797050 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a3cf-6d66-4d2e-ab7d-10e1f585cf5a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:14.797137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ee409-c5ee-4bfb-ae7a-a59bbf7d9329 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:14.797213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a3cf-6d66-4d2e-ab7d-10e1f585cf5a None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:14.798284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 117/224] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:14.798436 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 108/225] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:40:14.805708 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68505-4804-4962-bc88-ca52394b4134 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:14.805922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a24d-9820-458d-bac4-99db14bc2d53 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:14.806009 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68505-4804-4962-bc88-ca52394b4134 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:14.806122 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a24d-9820-458d-bac4-99db14bc2d53 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:14.806295 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68505-4804-4962-bc88-ca52394b4134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:14.806787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a24d-9820-458d-bac4-99db14bc2d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:14.825806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-0328a24d-9820-458d-bac4-99db14bc2d53 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:14.826008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-cfc68505-4804-4962-bc88-ca52394b4134 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:14.827009 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-0328a24d-9820-458d-bac4-99db14bc2d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:14.827142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-cfc68505-4804-4962-bc88-ca52394b4134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:14.829176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:14.898586 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 109/226] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:14.904647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 118/227] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:14.915353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0273d67-af7f-4839-a3f1-34948b25821b None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:14.915353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0273d67-af7f-4839-a3f1-34948b25821b None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:14.915353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0273d67-af7f-4839-a3f1-34948b25821b None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:14.915353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 110/228] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:14.919780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:14.920077 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:14.920357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:14.938419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:14.939637 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:15.007460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 119/229] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:15.173812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0273d67-af7f-4839-a3f1-34948b25821b None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.176297 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.220849 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.284674 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30200e6-beff-49e5-bcaa-d9cefae59363 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.287731 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0273d67-af7f-4839-a3f1-34948b25821b None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.318991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.319662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.319946 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.320226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.323007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.323672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.323957 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.324234 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.330113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:15.331775 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.335410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:15.335925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:15.337099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 120/230] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:15.338229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:15.342768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:15.344042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:15.345116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 111/231] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:15.348724 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.436538 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.439448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d8a5c-2a6b-44e1-9d60-4daff3345061 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.450121 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954079b-cacb-4e6c-83b1-c96d0adbaa4a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.474533 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.475167 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.475487 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.475770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.476896 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.478347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.479954 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 121/232] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:15.480713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.484168 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.484857 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.485140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.485420 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.486089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:15.486399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.488137 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.497418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 112/233] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:15.498204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.557560 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.580886 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455a395-40a3-481d-a8aa-b8ad79f78f60 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.599520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2091f6-3427-4461-b102-fff15a5f3bf1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.614406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.615011 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.615318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.615602 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.616298 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] RBAC: Adding query filter params (name=reader) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:15.616605 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.617975 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.625951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 122/234] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:15.626661 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.635724 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.636348 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.636638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.636920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] PATH_INFO: `/v3/groups` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.637558 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.639012 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.657264 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.724653 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8dd6c-a11e-4043-9531-383b30aa4cee None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.757075 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.757707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.757998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.758281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] PATH_INFO: `/v3/users/2bdd1d0d8fd240cfae77d3d5f822124c` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.759458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] RBAC: Authorizing `identity:get_user(user_id=2bdd1d0d8fd240cfae77d3d5f822124c)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.760928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.761928 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 113/235] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Oct 17 05:40:15 2022] POST /identity/v3/groups => generated 229 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:15.762826 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.768797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 123/236] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/users/2bdd1d0d8fd240cfae77d3d5f822124c => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:15.770263 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:15.887136 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.892116 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd681dc3-dbb9-47d0-ae58-92b502ec3384 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.892330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571cc20-4d30-4323-970c-b4ba8100863d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:15.919911 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:15.933994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd4845-0d78-4484-b62c-34605af737e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.934269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:15.934660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd4845-0d78-4484-b62c-34605af737e4 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.934837 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:15.934948 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd4845-0d78-4484-b62c-34605af737e4 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.935130 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:15.935238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd4845-0d78-4484-b62c-34605af737e4 None admin] PATH_INFO: `/v3/system/users/2bdd1d0d8fd240cfae77d3d5f822124c/roles/153c5cb975cc4d2b868b1cf6ce5331cc` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.935424 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] PATH_INFO: `/v3/users/5fef7b202a164684982df04ac54ff5b6` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:15.937876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd4845-0d78-4484-b62c-34605af737e4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2bdd1d0d8fd240cfae77d3d5f822124c, role_id=153c5cb975cc4d2b868b1cf6ce5331cc)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.938889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd4845-0d78-4484-b62c-34605af737e4 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.949779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] RBAC: Authorizing `identity:get_user(user_id=5fef7b202a164684982df04ac54ff5b6)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:15.951514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:15.953473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 114/237] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/users/5fef7b202a164684982df04ac54ff5b6 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:15.960327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:16.003838 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 124/238] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Oct 17 05:40:15 2022] PUT /identity/v3/system/users/2bdd1d0d8fd240cfae77d3d5f822124c/roles/153c5cb975cc4d2b868b1cf6ce5331cc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:40:16.068187 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced5c38-4e0c-45d0-bfe9-2120e04eb052 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:16.108129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:16.108761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:16.109044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:16.109320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:16.110333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] RBAC: Adding query filter params (role.id=77152c82ebb94862b2211a7783423231, user.id=5fef7b202a164684982df04ac54ff5b6, scope.system=all) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:16.110634 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:16.112147 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3e2a-5d36-441b-920e-d0633500d955 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:16.119425 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 115/239] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:40:15 2022] GET /identity/v3/role_assignments?user.id=5fef7b202a164684982df04ac54ff5b6&scope.system=all&role.id=77152c82ebb94862b2211a7783423231 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:17.964745 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2631d92-4fa4-43a1-b99e-11113e35887d None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:17.965323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2631d92-4fa4-43a1-b99e-11113e35887d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:17.965323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2631d92-4fa4-43a1-b99e-11113e35887d None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:17.966929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 125/240] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:17.972681 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea5fe-3a82-43e9-89c6-a40e00b93110 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:17.973099 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea5fe-3a82-43e9-89c6-a40e00b93110 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:17.973284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea5fe-3a82-43e9-89c6-a40e00b93110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:17.991814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-500ea5fe-3a82-43e9-89c6-a40e00b93110 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:17.993037 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-500ea5fe-3a82-43e9-89c6-a40e00b93110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:18.032586 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:18.068190 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 116/241] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:18.147174 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1719c5e-6e70-4587-b3cd-22f2480377f3 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.147548 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1719c5e-6e70-4587-b3cd-22f2480377f3 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.147750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1719c5e-6e70-4587-b3cd-22f2480377f3 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.149354 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 126/242] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:18.155406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d1031-7668-457a-a335-b8f5d4ce5eb2 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.155756 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d1031-7668-457a-a335-b8f5d4ce5eb2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.156382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d1031-7668-457a-a335-b8f5d4ce5eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.176954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-ff4d1031-7668-457a-a335-b8f5d4ce5eb2 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:18.178183 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-ff4d1031-7668-457a-a335-b8f5d4ce5eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:18.248540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 117/243] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:18.264712 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412850d-498f-487d-ac7b-578d7741ee8b None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.265000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412850d-498f-487d-ac7b-578d7741ee8b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.265286 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412850d-498f-487d-ac7b-578d7741ee8b None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.266770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 127/244] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:18.271738 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.272035 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.272314 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.290381 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:18.291599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:18.336007 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412850d-498f-487d-ac7b-578d7741ee8b None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.359508 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 118/245] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:18.443101 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412850d-498f-487d-ac7b-578d7741ee8b None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:18.476636 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:18.477275 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.477565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.477846 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.478961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:18.480464 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:18.482249 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 128/246] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:18.488256 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.519148 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:18.519587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.552953 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:18.595877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c9e41-a87a-4fbc-a903-3c366508b37c None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:18.621928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe258e2-ed4e-45c4-ad8c-faafa8c29c9d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:18.634359 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:18.635460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.635788 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.636077 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] PATH_INFO: `/v3/groups` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.636738 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:18.638193 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:18.655485 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:18.660759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.660759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.660759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.667612 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:18.673957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:18.676091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 129/247] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:18.677159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.757211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 119/248] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Oct 17 05:40:18 2022] POST /identity/v3/groups => generated 226 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:40:18.758304 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.774884 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51d8a-eb23-479b-8a1a-06fe7cc876a6 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:18.777283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:18.828791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:18.829446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.829770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.830006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.843152 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:18.849196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:18.850430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:18.851412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 130/249] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:18.856393 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.871617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:18.881532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b4fe6-d690-46e1-95f4-9eafeee4b2d1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:18.922863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:18.923526 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:18.923813 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:18.924089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:18.924792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] RBAC: Adding query filter params (name=member) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:18.925104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:18.926517 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:18.939722 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 120/250] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:18.943738 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:18.982058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f27ef-e241-4ebb-9bde-b283e2b55d9d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:19.029450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:19.030086 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:19.030377 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:19.030666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:19.031435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] RBAC: Adding query filter params (name=reader) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:19.031760 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:19.033184 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:19.043829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 131/251] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:19.049981 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:19.067411 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:19.074461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce61fb-70b4-42f0-b696-ea9951700e57 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:19.122921 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:19.123547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:19.123837 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:19.124859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] PATH_INFO: `/v3/users/2bdd1d0d8fd240cfae77d3d5f822124c` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:19.126330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] RBAC: Authorizing `identity:get_user(user_id=2bdd1d0d8fd240cfae77d3d5f822124c)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:19.127911 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:19.130346 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 121/252] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:18 2022] GET /identity/v3/users/2bdd1d0d8fd240cfae77d3d5f822124c => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:19.136080 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:19.172222 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:19.186821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3f74-9b63-4a35-8b34-0e1265a1b578 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:19.238344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:19.239723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:19.240046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:19.240301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] PATH_INFO: `/v3/users/5fef7b202a164684982df04ac54ff5b6` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:19.244026 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] RBAC: Authorizing `identity:get_user(user_id=5fef7b202a164684982df04ac54ff5b6)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:19.246172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:19.248439 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 132/253] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:19 2022] GET /identity/v3/users/5fef7b202a164684982df04ac54ff5b6 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:19.255007 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:19.276551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0e9b4-0bfb-435e-84ae-b566e353a2bb None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:19.316739 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:19.317363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:19.317655 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:19.317935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:19.318680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] RBAC: Adding query filter params (role.id=153c5cb975cc4d2b868b1cf6ce5331cc, user.id=2bdd1d0d8fd240cfae77d3d5f822124c, scope.system=all) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:19.318999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:19.320547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70d8dc-a740-48e9-a0c1-2c8ec9a6ac75 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:19.343753 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 122/254] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:40:19 2022] GET /identity/v3/role_assignments?user.id=2bdd1d0d8fd240cfae77d3d5f822124c&scope.system=all&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:19.367798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:19.387849 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787915fd-609a-4cfc-89d9-78ed7aff314f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:19.433410 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051942f9-3e02-46bb-9dde-cc7ea51973ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:19.434087 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051942f9-3e02-46bb-9dde-cc7ea51973ba None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:19.434376 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051942f9-3e02-46bb-9dde-cc7ea51973ba None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:19.434660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051942f9-3e02-46bb-9dde-cc7ea51973ba None admin] PATH_INFO: `/v3/system/users/5fef7b202a164684982df04ac54ff5b6/roles/77152c82ebb94862b2211a7783423231` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:19.436216 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051942f9-3e02-46bb-9dde-cc7ea51973ba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5fef7b202a164684982df04ac54ff5b6, role_id=77152c82ebb94862b2211a7783423231)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:19.437687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051942f9-3e02-46bb-9dde-cc7ea51973ba None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:19.546535 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 133/255] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Oct 17 05:40:19 2022] PUT /identity/v3/system/users/5fef7b202a164684982df04ac54ff5b6/roles/77152c82ebb94862b2211a7783423231 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:40:21.621268 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8b75f-8d14-47c2-9895-518e99cffaa2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:21.621842 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8b75f-8d14-47c2-9895-518e99cffaa2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:21.621842 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8b75f-8d14-47c2-9895-518e99cffaa2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:21.623472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 123/256] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:21.633947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628251d9-a46a-4d48-be18-c3935b29cbf7 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:21.634904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628251d9-a46a-4d48-be18-c3935b29cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:21.635460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628251d9-a46a-4d48-be18-c3935b29cbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:21.660831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-628251d9-a46a-4d48-be18-c3935b29cbf7 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:21.662457 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-628251d9-a46a-4d48-be18-c3935b29cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:21.732800 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 134/257] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:40:21.751133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc634280-9988-4397-a2c4-a9f2014ffb3d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:21.751476 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc634280-9988-4397-a2c4-a9f2014ffb3d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:21.751764 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc634280-9988-4397-a2c4-a9f2014ffb3d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:21.753682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 124/258] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:21.759429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:21.759996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:21.760523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:21.783572 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:21.785683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:21.812614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 135/259] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:40:22.105333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc634280-9988-4397-a2c4-a9f2014ffb3d None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.123066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc634280-9988-4397-a2c4-a9f2014ffb3d None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.127827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.132932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.132932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.132932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.142899 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:22.149291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:22.150393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:22.151355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 125/260] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:22.156695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.166624 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62113a2-93ec-40de-a6bb-f7389ccf7681 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.171905 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.174600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.174600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.174600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.174600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] RBAC: Adding query filter params (name=member) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:22.174600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.177210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: WARNING py.warnings [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:40:22.177210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: warnings.warn(msg) Oct 17 05:40:22.177210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: Oct 17 05:40:22.178498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.187956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 136/261] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:22.193788 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.204395 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5174a-3c13-44a2-850f-f5f814ee5936 admin admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.209587 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.210415 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.210857 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.211290 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.218470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.220600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: WARNING py.warnings [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:40:22.220600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: warnings.warn(msg) Oct 17 05:40:22.220600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: Oct 17 05:40:22.222824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.224899 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 126/262] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:22.230622 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.241033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886341ac-ee62-4f0d-9233-e6c4c1731f1d admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.244696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.250891 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.251597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.251883 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.253412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.256077 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.258133 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.258920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42504fb3-71ce-406f-9088-11d9f78aec96 admin admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.260194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 137/263] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:22.267922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.268926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.269454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.270171 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.273026 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.284888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:22.288942 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01438abd-86d9-43e2-8358-3b34ce684a80 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:22.290577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:22.291774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:22.292713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:22.297439 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 127/264] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:22.303851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.304531 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.304817 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.305106 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/153c5cb975cc4d2b868b1cf6ce5331cc` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.307730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1cdee355bb441ea1a53da3a457e54e, group_id=4270be876d954f6e8c8f1b2b0c60428f, role_id=153c5cb975cc4d2b868b1cf6ce5331cc)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.308335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.308767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: WARNING py.warnings [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:40:22.308767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: warnings.warn(msg) Oct 17 05:40:22.308767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: Oct 17 05:40:22.310864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.322146 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee9e34-dda6-4f14-95d4-9f5d52a9d395 admin admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.329124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.329981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.330453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.330993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.331914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:22.332429 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.333705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: WARNING py.warnings [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:40:22.333705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: warnings.warn(msg) Oct 17 05:40:22.333705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: Oct 17 05:40:22.335080 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.348241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 128/265] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:22.353638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.365440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814184f-48a6-4d64-ba4f-10a25d127b32 admin admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.370956 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.372101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.372632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.373123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.374779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.377380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.380731 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 129/266] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:22.388585 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.391761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 138/267] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:40:22 2022] PUT /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/153c5cb975cc4d2b868b1cf6ce5331cc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:22.404211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f649ddd-d559-47cc-bac0-14958ffb21dd admin admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.455763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:22.492108 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee25e9c-b16b-47a6-a2bc-0f1d02c30f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.493168 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee25e9c-b16b-47a6-a2bc-0f1d02c30f16 admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.494368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee25e9c-b16b-47a6-a2bc-0f1d02c30f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.494637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee25e9c-b16b-47a6-a2bc-0f1d02c30f16 admin admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.496053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee25e9c-b16b-47a6-a2bc-0f1d02c30f16 admin admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.498323 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee25e9c-b16b-47a6-a2bc-0f1d02c30f16 admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.500580 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 130/268] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:40:22 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:22.506677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:22.527113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744a9f1-77a0-4a9e-988e-34770e73a29a admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:22.540105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8f682-692f-49dd-9753-fdd9baf8b94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:22.540803 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8f682-692f-49dd-9753-fdd9baf8b94e admin admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.541089 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8f682-692f-49dd-9753-fdd9baf8b94e admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.541370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8f682-692f-49dd-9753-fdd9baf8b94e admin admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/c040847a15b148f893d961a9a578db29` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.543665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8f682-692f-49dd-9753-fdd9baf8b94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1cdee355bb441ea1a53da3a457e54e, group_id=4270be876d954f6e8c8f1b2b0c60428f, role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:22.546596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8f682-692f-49dd-9753-fdd9baf8b94e admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:22.620628 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 139/269] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:40:22 2022] PUT /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/c040847a15b148f893d961a9a578db29 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:22.671689 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb2f6a-74b4-4414-93da-a0712aac2b57 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.672221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb2f6a-74b4-4414-93da-a0712aac2b57 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.672305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb2f6a-74b4-4414-93da-a0712aac2b57 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.674006 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 131/270] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:22.679964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644a571-0c43-405f-89c4-426f8e1eeef9 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.680453 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644a571-0c43-405f-89c4-426f8e1eeef9 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.680536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644a571-0c43-405f-89c4-426f8e1eeef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.700987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9583821-81bc-46a9-8358-da64146b66bd None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.701223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9583821-81bc-46a9-8358-da64146b66bd None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.701489 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9583821-81bc-46a9-8358-da64146b66bd None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.703151 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 132/271] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:40:22.705019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-6644a571-0c43-405f-89c4-426f8e1eeef9 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:22.706772 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-6644a571-0c43-405f-89c4-426f8e1eeef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:22.709726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0bc97-8cfc-45b2-9dd1-df3e4ec0f1e2 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:22.710028 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0bc97-8cfc-45b2-9dd1-df3e4ec0f1e2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:22.710313 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0bc97-8cfc-45b2-9dd1-df3e4ec0f1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:22.739105 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-74f0bc97-8cfc-45b2-9dd1-df3e4ec0f1e2 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:22.740673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-74f0bc97-8cfc-45b2-9dd1-df3e4ec0f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:22.784991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:22.835458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 140/272] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:40:22.878092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 133/273] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:23.086577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0364-ee46-4233-bc7c-85585cde0936 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.087272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0364-ee46-4233-bc7c-85585cde0936 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.087810 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0364-ee46-4233-bc7c-85585cde0936 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.089796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 141/274] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:23.096353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.096680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.096964 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.125233 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:23.132706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:23.146321 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 134/275] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:40:23.397705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0364-ee46-4233-bc7c-85585cde0936 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.412008 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.415626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0364-ee46-4233-bc7c-85585cde0936 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.435485 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.436132 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.436415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.436736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.460718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:23.465620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:23.466645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:23.467632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:23.472966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 142/276] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:23.477451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.506219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4053325-928f-4f67-8a16-00d0d540cf78 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.520306 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.520928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.521211 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.521487 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.522156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] RBAC: Adding query filter params (name=member) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:23.522452 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:23.524100 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:23.547460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 143/277] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:23.553936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.570995 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f1ff6-4e91-40a2-ae9f-38ce85513aa8 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.581495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.582144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.582434 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.582712 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.587475 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:23.588517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: WARNING py.warnings [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 05:40:23.588517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: warnings.warn(msg) Oct 17 05:40:23.588517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: Oct 17 05:40:23.589745 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:23.591391 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 144/278] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:23.599049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.612788 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406d3e3-5ec1-4c81-9515-54a93ee4f15d admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.616006 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb795447-0e15-468b-85b3-45e506cf9019 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.628098 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.628751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.629046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.629332 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.631778 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:23.668964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:23.669983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:23.671012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 145/279] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 74 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:23.672978 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.684831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c732e1-bdc5-46d9-8f05-e041ff36e3ac admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.694509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.695230 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.695585 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.695823 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.699291 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.699936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.700226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.700509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.702710 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:23.704875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:23.709768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 146/280] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:23.717588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.727329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:23.732536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:23.735016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:23.737335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 135/281] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:23.742293 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.747890 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c6726-4714-4a8a-acbb-661c125e9540 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.759711 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.760337 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.760633 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.760918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.761607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:23.761926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:23.763575 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:23.777106 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 147/282] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:23.777953 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.804530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db6f95-f138-4dbe-8cd8-e9a9156ebc7f admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.820000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.820684 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.820972 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.821256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/153c5cb975cc4d2b868b1cf6ce5331cc` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.825294 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ad5db5d05144b98ec29f5c46a876a7, group_id=4270be876d954f6e8c8f1b2b0c60428f, role_id=153c5cb975cc4d2b868b1cf6ce5331cc)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:23.832067 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:23.902658 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddfcb9-b48e-42a8-a28a-b1066e48d773 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:23.939975 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 148/283] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:40:23 2022] PUT /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/153c5cb975cc4d2b868b1cf6ce5331cc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:23.941972 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:23.946615 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:23.947392 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.947796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.948083 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.948773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:23.949085 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:23.950889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e546bb-c19a-4f6d-9988-f670004acda4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:23.963355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 136/284] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:23.964831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3283f-182e-4faa-b4e0-a61cadcd1cd3 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:23.965125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3283f-182e-4faa-b4e0-a61cadcd1cd3 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:23.965403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3283f-182e-4faa-b4e0-a61cadcd1cd3 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:23.971105 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 137/285] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:23.973156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3283f-182e-4faa-b4e0-a61cadcd1cd3 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.017136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:24.086739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465a964-03cc-40db-96eb-97640768bedb admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.094863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef463c-2fd8-438c-8826-2e7dda94c10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.095554 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef463c-2fd8-438c-8826-2e7dda94c10f admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.096278 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef463c-2fd8-438c-8826-2e7dda94c10f admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.096278 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef463c-2fd8-438c-8826-2e7dda94c10f admin admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.097305 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef463c-2fd8-438c-8826-2e7dda94c10f admin admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.099315 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef463c-2fd8-438c-8826-2e7dda94c10f admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.101153 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 149/286] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:24.102162 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.102454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.102734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.114435 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3283f-182e-4faa-b4e0-a61cadcd1cd3 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.152241 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:24.153471 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:24.175667 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 150/287] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:40:24.176550 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.187402 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196039-d081-49bc-b4f0-c3349ffa789c None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.198268 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.198843 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.199134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.199421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.204006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.206635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.208518 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 151/288] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:40:24 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:24.209465 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.210138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.210431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.210716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] PATH_INFO: `/v3/users/5fef7b202a164684982df04ac54ff5b6` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.213984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.216013 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=5fef7b202a164684982df04ac54ff5b6)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.217504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.219454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 138/289] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:40:23 2022] GET /identity/v3/users/5fef7b202a164684982df04ac54ff5b6 => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:24.225387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.227825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8cc8f-d924-47a8-ba3c-1f919dad5be1 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.235048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.235798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.236083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.236362 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/c040847a15b148f893d961a9a578db29` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.239206 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ad5db5d05144b98ec29f5c46a876a7, group_id=4270be876d954f6e8c8f1b2b0c60428f, role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.242251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.311163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 152/290] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:40:24 2022] PUT /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7/groups/4270be876d954f6e8c8f1b2b0c60428f/roles/c040847a15b148f893d961a9a578db29 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:40:24.388929 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:24.396465 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7e8eb-17ea-4ac4-919f-065c6e477ee2 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.436564 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.437191 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.437477 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.437757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.438506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] RBAC: Adding query filter params (role.id=77152c82ebb94862b2211a7783423231, user.id=5fef7b202a164684982df04ac54ff5b6, scope.system=all) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:24.438822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.440406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.459864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 139/291] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 05:40:24 2022] GET /identity/v3/role_assignments?user.id=5fef7b202a164684982df04ac54ff5b6&scope.system=all&role.id=77152c82ebb94862b2211a7783423231 => generated 510 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:24.680790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.761874 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529149d5-35b8-4d55-beb3-e20094f62b02 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.768161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.768909 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.769201 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.769482 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.771655 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:24.781396 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:24.787514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:24.788536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:24.789495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 153/292] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:40:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:24.797128 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.812565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9b7b-a292-4ff6-84a9-3cd35d445b5f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.820908 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.821556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.821845 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.822131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.822826 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:24.823144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.824907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.837254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 140/293] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:40:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:24.842538 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.853675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2638bf-a44d-4923-8d56-cdda140db4f7 admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.859913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.860565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.860854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.861135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] PATH_INFO: `/v3/groups/fd9b9734a8594ef9a30a7caaa0f278b7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.872509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] RBAC: Authorizing `identity:get_group(group_id=fd9b9734a8594ef9a30a7caaa0f278b7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.874486 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.876547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 154/294] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 05:40:24 2022] GET /identity/v3/groups/fd9b9734a8594ef9a30a7caaa0f278b7 => generated 229 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:24.881940 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.894469 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b7ece-ad37-4049-9684-356a412348ef admin admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.900743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64402108-ccb4-4b12-a3d1-9fee5018b89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.901398 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64402108-ccb4-4b12-a3d1-9fee5018b89a admin admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.901687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64402108-ccb4-4b12-a3d1-9fee5018b89a admin admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.901966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64402108-ccb4-4b12-a3d1-9fee5018b89a admin admin] PATH_INFO: `/v3/projects/be40c9d78f744db094612526ea19176d` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.903142 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64402108-ccb4-4b12-a3d1-9fee5018b89a admin admin] RBAC: Authorizing `identity:get_project(project_id=be40c9d78f744db094612526ea19176d)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.905045 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64402108-ccb4-4b12-a3d1-9fee5018b89a admin admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:24.906818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 141/295] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:40:24 2022] GET /identity/v3/projects/be40c9d78f744db094612526ea19176d => generated 344 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:24.912505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:24.924461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136760f-fbee-44de-9305-656ab1bb933e admin admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:24.930142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564e07b-f426-45ad-bb12-08bacfe42cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be40c9d78f744db094612526ea19176d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:24.930833 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564e07b-f426-45ad-bb12-08bacfe42cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:24.931126 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564e07b-f426-45ad-bb12-08bacfe42cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:24.931458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564e07b-f426-45ad-bb12-08bacfe42cd5 admin admin] PATH_INFO: `/v3/projects/be40c9d78f744db094612526ea19176d/groups/fd9b9734a8594ef9a30a7caaa0f278b7/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:24.933679 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564e07b-f426-45ad-bb12-08bacfe42cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be40c9d78f744db094612526ea19176d, group_id=fd9b9734a8594ef9a30a7caaa0f278b7, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:24.936415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564e07b-f426-45ad-bb12-08bacfe42cd5 admin admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:25.009686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 155/296] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 05:40:24 2022] PUT /identity/v3/projects/be40c9d78f744db094612526ea19176d/groups/fd9b9734a8594ef9a30a7caaa0f278b7/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:25.561380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd2ac-8a45-4b56-8159-db9da960b531 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:25.561795 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd2ac-8a45-4b56-8159-db9da960b531 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:25.562000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd2ac-8a45-4b56-8159-db9da960b531 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:25.563637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 142/297] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:25.569586 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4cefc-6dbb-4d6d-9f2b-a005548d9d86 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:25.569889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4cefc-6dbb-4d6d-9f2b-a005548d9d86 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:25.570178 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4cefc-6dbb-4d6d-9f2b-a005548d9d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:25.595581 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-faa4cefc-6dbb-4d6d-9f2b-a005548d9d86 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:25.596846 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-faa4cefc-6dbb-4d6d-9f2b-a005548d9d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:25.687987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 156/298] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:25.847425 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f407b-3169-4058-bf47-a361bd3d8d04 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:25.847980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f407b-3169-4058-bf47-a361bd3d8d04 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:25.847980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f407b-3169-4058-bf47-a361bd3d8d04 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:25.849603 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 143/299] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:25.854809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:25.855236 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:25.855478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:25.880426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:25.882022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:25.974848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 157/300] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:26.047084 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f407b-3169-4058-bf47-a361bd3d8d04 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:26.185066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f407b-3169-4058-bf47-a361bd3d8d04 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:26.230325 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:26.230973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:26.231253 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:26.231567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:26.239750 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:26.246223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:26.251125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:26.252176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:26.253678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 144/301] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:26.260232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:26.335984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:26.356830 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:26.421573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e4fd0-b08d-4dec-8e61-e25718fd573a None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:26.472491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:26.473035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:26.473323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:26.473606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:26.474293 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] RBAC: Adding query filter params (name=member) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:26.474609 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:26.476305 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:26.487082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 158/302] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:26.500473 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:26.584251 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1fd1a-2160-447e-ada9-4c23a95b0ee3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:26.662403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:26.663136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:26.663455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:26.663805 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:26.685254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:26.692079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:26.693117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:26.694076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:26.694782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 145/303] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:40:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:26.696021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:26.704378 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:26.709557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a6f21-9785-46fc-a228-f8ad93be4ce1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:26.787249 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d95b1-fa17-4127-9d02-b3c91c26a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:26.787830 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d95b1-fa17-4127-9d02-b3c91c26a351 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:26.788114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d95b1-fa17-4127-9d02-b3c91c26a351 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:26.788395 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d95b1-fa17-4127-9d02-b3c91c26a351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:26.795892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:26.848263 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5f107-7703-44b4-a8dc-e6ba8a755604 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:26.890901 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:26.891573 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:26.891862 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:26.892143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:26.892823 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:26.893132 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:26.894659 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175da27f-d639-4f49-85e8-0a7b44060b1a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:26.905991 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 146/304] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:26.913270 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b829-e8e9-4b3f-a59e-d767e38661a6 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:26.913589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b829-e8e9-4b3f-a59e-d767e38661a6 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:26.913879 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836b829-e8e9-4b3f-a59e-d767e38661a6 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:26.915666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 147/305] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:26.916433 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b829-e8e9-4b3f-a59e-d767e38661a6 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:26.936791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d95b1-fa17-4127-9d02-b3c91c26a351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:26.939072 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d95b1-fa17-4127-9d02-b3c91c26a351 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:27.054498 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836b829-e8e9-4b3f-a59e-d767e38661a6 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:27.061054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:27.069005 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 159/306] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:27.070851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.071358 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.075889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.083504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:27.091261 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:27.093960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:27.097852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:27.102863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.102863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.102863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.181660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 160/307] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:27.182866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:27.201256 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:27.203139 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:27.374304 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2b70f-13f3-416e-8af1-5b0b0e556fe7 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:27.398916 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 148/308] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:27.413721 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:27.424143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:27.449333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:27.449856 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.450147 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.453335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.453711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:27.456300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:27.457837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:27.460000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 161/309] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:27 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:27.465289 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:27.544494 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ff82a-e9dd-4c98-b38b-8c1de99becc4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:27.588905 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c5cf6-4f44-451d-a57a-36bf289fa097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:27.589555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c5cf6-4f44-451d-a57a-36bf289fa097 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.589835 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c5cf6-4f44-451d-a57a-36bf289fa097 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.590114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c5cf6-4f44-451d-a57a-36bf289fa097 None admin] PATH_INFO: `/v3/projects/8a1cdee355bb441ea1a53da3a457e54e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.592678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c5cf6-4f44-451d-a57a-36bf289fa097 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1cdee355bb441ea1a53da3a457e54e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:27.594140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c5cf6-4f44-451d-a57a-36bf289fa097 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:27.596156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 149/310] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:27 2022] GET /identity/v3/projects/8a1cdee355bb441ea1a53da3a457e54e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:27.597897 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b49a3-a892-4122-9697-53e003ea36dc None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.598186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b49a3-a892-4122-9697-53e003ea36dc None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.598461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b49a3-a892-4122-9697-53e003ea36dc None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.599699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 150/311] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:27.604194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b49a3-a892-4122-9697-53e003ea36dc None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:27.626064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0ac78-15d8-4ccf-b244-4495d0b99094 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:27.671902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da81e9-f9e9-4cce-9cfd-57547dc01fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:27.672549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da81e9-f9e9-4cce-9cfd-57547dc01fd2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.672828 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da81e9-f9e9-4cce-9cfd-57547dc01fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.673104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da81e9-f9e9-4cce-9cfd-57547dc01fd2 None admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.674324 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da81e9-f9e9-4cce-9cfd-57547dc01fd2 None admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:27.675987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da81e9-f9e9-4cce-9cfd-57547dc01fd2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:27.678117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 162/312] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:40:27 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:27.679172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.679555 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.679850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.703888 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:27.705605 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:27.732041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:27.746900 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b49a3-a892-4122-9697-53e003ea36dc None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:27.784867 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:27.797062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:27.799357 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:27.799555 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 163/313] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:27.799555 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:27.803668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:27.803966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:27.809660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:27.811122 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:27.812942 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 151/314] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:40:27 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:27.815662 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:27.949681 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b932a-ea58-4d51-94cd-90cb06dd60f3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:27.950284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c109176-674d-43fb-9b50-b635cff9a7ae None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.000660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b228-74ea-4257-becc-9bedc52df376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.000660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b228-74ea-4257-becc-9bedc52df376 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.000660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b228-74ea-4257-becc-9bedc52df376 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.000660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b228-74ea-4257-becc-9bedc52df376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.001107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.001107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.001107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.001107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.004250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] RBAC: Adding query filter params (group.id=4270be876d954f6e8c8f1b2b0c60428f, role.id=153c5cb975cc4d2b868b1cf6ce5331cc, scope.project.id=8a1cdee355bb441ea1a53da3a457e54e) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:28.004250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:28.004746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:28.011530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1b228-74ea-4257-becc-9bedc52df376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:28.012533 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 152/315] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:40:27 2022] GET /identity/v3/role_assignments?group.id=4270be876d954f6e8c8f1b2b0c60428f&scope.project.id=8a1cdee355bb441ea1a53da3a457e54e&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:28.013372 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-e5d1b228-74ea-4257-becc-9bedc52df376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:28.021691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:28.022870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:40:28.023962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:28.024689 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 164/316] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:28.027076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af3b07-8b44-464d-a3e0-d9abe66c75c6 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.027076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af3b07-8b44-464d-a3e0-d9abe66c75c6 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.027076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af3b07-8b44-464d-a3e0-d9abe66c75c6 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.030742 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 165/317] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:28.032676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af3b07-8b44-464d-a3e0-d9abe66c75c6 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.051676 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.111254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.161852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4fe32-bfe4-4f19-944f-04363e6e33e9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.177278 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af3b07-8b44-464d-a3e0-d9abe66c75c6 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.210366 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.211361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.211877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.212373 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.213309 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] RBAC: Adding query filter params (group.id=4270be876d954f6e8c8f1b2b0c60428f, role.id=c040847a15b148f893d961a9a578db29, scope.project.id=8a1cdee355bb441ea1a53da3a457e54e) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:28.213845 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:28.215806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af3337-a21c-4fd2-b0b1-6335d4b427d4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:28.223901 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.224555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 153/318] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/role_assignments?group.id=4270be876d954f6e8c8f1b2b0c60428f&scope.project.id=8a1cdee355bb441ea1a53da3a457e54e&role.id=c040847a15b148f893d961a9a578db29 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:28.224817 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.225117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.225400 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.225873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.226085 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] RBAC: Adding query filter params (name=member) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:28.226205 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.226405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:28.226518 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.232068 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:28.242533 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 166/319] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:28.243621 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.253442 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:28.255073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:28.350515 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 154/320] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:28.351596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.382797 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.386236 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ee87-663a-476a-b8b3-e4272d375e80 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.407269 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.428387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.429105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.429394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.429673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.444190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:28.450592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:28.451701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:28.452632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 167/321] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:28.462370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.497118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834cb9a-3227-47a4-bef5-49ff3525d022 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.549167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.549871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.550227 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.550664 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.597063 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4f3b5-2972-485f-ac43-8f0598386dc2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.634983 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:28.636522 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:28.638281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.638834 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.639117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.640105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.640105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:28.640410 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:28.641732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:28.652884 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 168/322] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:28.653724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.658132 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.698083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.765500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 155/323] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:28.766112 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.773880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb64f3-faac-4fae-a3f6-e685b26f7d02 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.813394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.814127 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.814422 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.814767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.833787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:28.838750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:28.839863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:28.840829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:28.841559 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 169/324] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:28.841634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:28.892259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38213ae-7499-4e07-a700-a57dca35a82e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:28.912818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.934262 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:28.934937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:28.935174 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:28.935495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:28.963596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:28.967897 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20129631-b5b9-4740-b6ce-6b5ce6bf9ad5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.015241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.016715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.016715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.016715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.017765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.017850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.018267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.019803 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.021650 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 170/325] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.022274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.122638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 156/326] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:29.123553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.129212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb6eb6-ea0e-4f7d-8e7c-3cfda52cd93b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.143007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.163928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.164544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.164831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.165111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.165782 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:29.166115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.167538 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.176911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 171/327] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.177707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.203922 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.229898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e5df4-8fe5-4b86-b5d9-0aaa45f6b32c None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.264044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.264680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.264970 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.265253 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.266379 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.267937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.269498 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 157/328] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.270266 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.282212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53102463-2d26-40ee-a175-89a9f1441a04 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.317037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.317655 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.317935 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.318211 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] PATH_INFO: `/v3/projects/73ad5db5d05144b98ec29f5c46a876a7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.319356 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] RBAC: Authorizing `identity:get_project(project_id=73ad5db5d05144b98ec29f5c46a876a7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.320788 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.322402 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 172/329] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/projects/73ad5db5d05144b98ec29f5c46a876a7 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.323150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.355439 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.373538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3055e8e-94b9-4834-a96c-6c15d72e24e1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.409950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.410558 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.410844 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.411123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.419213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.426561 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970944e0-3f18-4b75-9c17-0a29549ceb42 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.460596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.460990 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.461282 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.461568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.462333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] RBAC: Adding query filter params (group.id=4270be876d954f6e8c8f1b2b0c60428f, role.id=153c5cb975cc4d2b868b1cf6ce5331cc, scope.project.id=73ad5db5d05144b98ec29f5c46a876a7) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:29.462656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.464123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.469910 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 173/330] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/role_assignments?group.id=4270be876d954f6e8c8f1b2b0c60428f&scope.project.id=73ad5db5d05144b98ec29f5c46a876a7&role.id=153c5cb975cc4d2b868b1cf6ce5331cc => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.470689 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.478930 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.480488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.581116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.598741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b18f9a-4732-4108-8447-6327951f5a71 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.601170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 158/331] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:29.607227 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.641255 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.642060 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.642165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.642443 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] PATH_INFO: `/v3/groups/4270be876d954f6e8c8f1b2b0c60428f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.643687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] RBAC: Authorizing `identity:get_group(group_id=4270be876d954f6e8c8f1b2b0c60428f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.645161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.646934 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 174/332] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/groups/4270be876d954f6e8c8f1b2b0c60428f => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.652942 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.687348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.734968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead8bde-3ffc-420a-b09c-615fb44b7120 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.788588 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.789632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.789787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e636e7-6b34-4772-bf60-052e093faf4b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.789895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.789895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] PATH_INFO: `/v3/projects/be40c9d78f744db094612526ea19176d` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.795128 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] RBAC: Authorizing `identity:get_project(project_id=be40c9d78f744db094612526ea19176d)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.796915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.798946 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 159/333] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/projects/be40c9d78f744db094612526ea19176d => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:29.805156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:29.836923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.838565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.839025 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.840013 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.840764 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] RBAC: Adding query filter params (group.id=4270be876d954f6e8c8f1b2b0c60428f, role.id=c040847a15b148f893d961a9a578db29, scope.project.id=73ad5db5d05144b98ec29f5c46a876a7) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:29.841220 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.843816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.850741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 175/334] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/role_assignments?group.id=4270be876d954f6e8c8f1b2b0c60428f&scope.project.id=73ad5db5d05144b98ec29f5c46a876a7&role.id=c040847a15b148f893d961a9a578db29 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:29.872592 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:29.924067 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807b8bd-02cb-4ba4-9109-33a43dc04ddb None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:29.960792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca02b03-42ba-48d2-b148-39b3d9b99188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:29.961461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca02b03-42ba-48d2-b148-39b3d9b99188 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:29.961757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca02b03-42ba-48d2-b148-39b3d9b99188 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:29.962042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca02b03-42ba-48d2-b148-39b3d9b99188 None admin] PATH_INFO: `/v3/groups/fd9b9734a8594ef9a30a7caaa0f278b7` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:29.963191 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca02b03-42ba-48d2-b148-39b3d9b99188 None admin] RBAC: Authorizing `identity:get_group(group_id=fd9b9734a8594ef9a30a7caaa0f278b7)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:29.964707 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca02b03-42ba-48d2-b148-39b3d9b99188 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:29.966344 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 160/335] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/groups/fd9b9734a8594ef9a30a7caaa0f278b7 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:29.972958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:30.070037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:30.079780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8d93-04e2-4cd2-bb25-9f67cea2fe4b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:30.114533 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:30.117524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:30.117524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:30.117524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:30.117524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] RBAC: Adding query filter params (group.id=fd9b9734a8594ef9a30a7caaa0f278b7, role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=be40c9d78f744db094612526ea19176d) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:30.117524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:30.118962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef9608-fb68-4783-a735-2601ca92922c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:30.124966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 176/336] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 05:40:29 2022] GET /identity/v3/role_assignments?group.id=fd9b9734a8594ef9a30a7caaa0f278b7&scope.project.id=be40c9d78f744db094612526ea19176d&role.id=df4da7cff6734799a09e0b770b60845f => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:32.741420 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ed7a-04a2-4f03-823c-8b543a7bb4d2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:32.742000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ed7a-04a2-4f03-823c-8b543a7bb4d2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:32.742000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ed7a-04a2-4f03-823c-8b543a7bb4d2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:32.743733 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 161/337] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:32.748728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30af48-c38f-461f-9b6b-4f600e472c4d None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:32.749134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30af48-c38f-461f-9b6b-4f600e472c4d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:32.749357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30af48-c38f-461f-9b6b-4f600e472c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:32.771340 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-6a30af48-c38f-461f-9b6b-4f600e472c4d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:32.772919 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-6a30af48-c38f-461f-9b6b-4f600e472c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:32.874199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 177/338] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:33.055411 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5e51-8a8f-4764-beef-3192bd4e0315 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.055965 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5e51-8a8f-4764-beef-3192bd4e0315 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.056429 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5e51-8a8f-4764-beef-3192bd4e0315 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.058185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 162/339] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:33.063906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.064471 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.065539 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.087050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:33.088606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:33.119204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11903-dabe-459d-98d9-9f73a75b734d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.119864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11903-dabe-459d-98d9-9f73a75b734d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.120399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11903-dabe-459d-98d9-9f73a75b734d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.122223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 163/340] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:33.127943 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.134120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.134120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.168316 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:33.169831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:33.208666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 178/341] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:33.256890 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:33.274666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:33.308213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 164/342] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:33.388420 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8ac4-04ed-4e42-9fe9-248e18b2994d None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:33.435356 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:33.436278 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.436731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.437174 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.447760 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:33.449431 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:33.476373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:33.477825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:33.479123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:33.479123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:33.479123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:33.481131 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 179/343] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:33.487291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:33.538628 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:33.559746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:33.615698 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:33.627860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5213f8-74f0-41b8-8948-c0a7bf4ba66a None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:33.652601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d28fc-568a-41ed-81f2-335a09d41ed5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:33.664642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83935e92-f322-4185-8e70-803327074d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:33.665267 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935e92-f322-4185-8e70-803327074d24 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.665553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935e92-f322-4185-8e70-803327074d24 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.665829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935e92-f322-4185-8e70-803327074d24 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.666492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83935e92-f322-4185-8e70-803327074d24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:33.666791 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83935e92-f322-4185-8e70-803327074d24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:33.668184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83935e92-f322-4185-8e70-803327074d24 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:33.681129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 165/344] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:33.682700 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82aed19-6daf-4908-a4e4-6e976a107cfa None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.683170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82aed19-6daf-4908-a4e4-6e976a107cfa None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.683639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82aed19-6daf-4908-a4e4-6e976a107cfa None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.689351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 166/345] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:33.689351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82aed19-6daf-4908-a4e4-6e976a107cfa None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:33.693504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:33.694006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.694288 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.694564 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.708282 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:33.709977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-338fe358-7618-4fa7-9865-708ce4b1ab1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:33.725444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:33.726494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:33.727518 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:33.727518 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:33.727518 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:33.729314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 180/346] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:33.730654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.731141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.731645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.751089 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:33.753626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:33.809376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82aed19-6daf-4908-a4e4-6e976a107cfa None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:33.821324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:33.843325 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 181/347] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:33.843325 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:33.860861 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558918f-5607-420c-a122-cac753a90200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:33.862398 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558918f-5607-420c-a122-cac753a90200 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.863255 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558918f-5607-420c-a122-cac753a90200 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.863930 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558918f-5607-420c-a122-cac753a90200 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.864720 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558918f-5607-420c-a122-cac753a90200 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:33.866425 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558918f-5607-420c-a122-cac753a90200 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:33.886576 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:33.938385 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675de72b-f845-4b72-a555-cc9fc5af972f None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:33.972413 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:33.973331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:33.974969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:33.975363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:33.982815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:33.982815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:33.982815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:33.988005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 167/348] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:33 2022] POST /identity/v3/users => generated 246 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:33.989007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558918f-5607-420c-a122-cac753a90200 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.000076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 182/349] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:34.005692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.134647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3fd0f-31bd-4ddc-a8ab-48a4874da9f2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:34.154823 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:34.184516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:34.185633 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.186172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.186675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.187707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:34.189571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:34.191248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558918f-5607-420c-a122-cac753a90200 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:34.232196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:34.232838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.233123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.233399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.245344 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:34.246887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:34.262197 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 183/350] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:34 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:34.267557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:34.267699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:34.268907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:34.269909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:34.269909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:34.269909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:34.269909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 168/351] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:34.276608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.319562 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:34.420587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff134-0fb9-43f7-8d82-1ca0689c8978 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:34.430272 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887521-9dcd-4d19-903b-ca0e1fb39520 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:34.463549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:34.464194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.464478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.464755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.475279 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:34.475926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.476218 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.476490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.477599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:34.478005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:34.478320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:34.479731 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410741d-47b4-4855-a734-9d6769c39006 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:34.480049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:34.493340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 169/352] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:34.494953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8b51f-3590-4e57-8a20-f6e0b59d3f41 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.495248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8b51f-3590-4e57-8a20-f6e0b59d3f41 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.495563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8b51f-3590-4e57-8a20-f6e0b59d3f41 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.497086 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 170/353] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:34.500185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:34.501353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:34.503091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:34.503091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:34.503091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:34.504374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.504665 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.504929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 184/354] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:34.505055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.506022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.529616 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:34.530969 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:34.533524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:34.620473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 171/355] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:34.621389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.641417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:34.651645 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9c2a0-349b-424c-976c-62290c21fad5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:34.736110 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:34.737518 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.737987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.738437 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.739562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:34.741189 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:34.780715 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bcc22-4900-48a2-b61c-5905316413e8 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:34.819466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:34.820138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:34.820386 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:34.820632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:34.821304 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:34.821594 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:34.822968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:34.835684 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 172/356] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:34.841141 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:34.872097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 185/357] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:34 2022] POST /identity/v3/users => generated 248 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:34.950973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:34.985620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:35.006362 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565a8e3-f67a-4c22-97fd-e21ed0742de3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:35.076208 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:35.077245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:35.078623 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:35.079292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:35.081643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:35.083399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:35.115626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:35.147840 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4e67-7eb5-4140-b664-01c9275c76e3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:35.191638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:35.192565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:35.193062 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:35.193523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:35.194389 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:35.196042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:35.236869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.sql.core [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 05:40:35.236869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 05:40:35.236869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [parameters: {'id': '5083609dadf34c069352b33a39ef9849', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 05:40:35.236869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=153118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result.read() Oct 17 05:40:35.255212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:40:35.256669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application yield resource Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise self.value Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application session.commit() Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:40:35.257796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:40:35.258809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:40:35.260064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result.read() Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:40:35.261207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [parameters: {'id': '5083609dadf34c069352b33a39ef9849', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:35.262384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 05:40:35.263519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:35.266382 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 186/358] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:40:34 2022] POST /identity/v3/roles => generated 144 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:35.278665 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 173/359] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:34 2022] POST /identity/v3/users => generated 249 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:40:35.279473 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:35.417995 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc96ec-f1c2-481a-8465-3288e853927d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:35.465441 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07f503-dad2-4c34-9726-e9641855d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:35.466113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07f503-dad2-4c34-9726-e9641855d452 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:35.466405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07f503-dad2-4c34-9726-e9641855d452 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:35.466688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07f503-dad2-4c34-9726-e9641855d452 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:35.476871 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:35.492804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07f503-dad2-4c34-9726-e9641855d452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8c07f503-dad2-4c34-9726-e9641855d452 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:35.500069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:35.501240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:35.502187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 187/360] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:40:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:35.508382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:35.669923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09058fb-85f1-4d6b-9a98-ed6363887ae9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:35.695228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:35.707901 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:35.709543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:35.709543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:35.709543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:35.710164 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:35.710485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:35.711919 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd778d-96bb-49e5-8533-9957f9bc5fb5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:35.722373 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 174/361] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:36.412152 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959afede-c80a-46f8-8120-c1a218b8de1b None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:36.412692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959afede-c80a-46f8-8120-c1a218b8de1b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:36.412692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959afede-c80a-46f8-8120-c1a218b8de1b None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:36.414397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 188/362] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:36.419465 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d763d-8759-4a37-8361-434dada27be5 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:36.419886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d763d-8759-4a37-8361-434dada27be5 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:36.420124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d763d-8759-4a37-8361-434dada27be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:36.443965 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-467d763d-8759-4a37-8361-434dada27be5 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:36.445185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-467d763d-8759-4a37-8361-434dada27be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:36.546159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 175/363] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:36.724266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267cdfc-8602-4cd1-bc61-4cc668c38cc4 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:36.724450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267cdfc-8602-4cd1-bc61-4cc668c38cc4 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:36.724744 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267cdfc-8602-4cd1-bc61-4cc668c38cc4 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:36.726108 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 189/364] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:36.740683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:36.740979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:36.741261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:36.764086 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:36.765272 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:36.867953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 176/365] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:37.066333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267cdfc-8602-4cd1-bc61-4cc668c38cc4 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.204918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267cdfc-8602-4cd1-bc61-4cc668c38cc4 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:37.242801 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:37.243836 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.244350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.244847 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.256170 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:37.261866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:37.262881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:37.263899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:37.266595 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 190/366] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:37.272402 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.316760 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.345816 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:37.401721 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:37.423802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6b56-2d03-45a2-98a2-5151efcba2b2 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:37.463454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:37.464438 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.464930 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.465500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.466836 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] RBAC: Adding query filter params (name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:37.467640 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:37.469224 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c310e-dbf8-4c70-9f5a-48d14a855c4a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:37.473370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c2b0-735f-4b2b-8776-3435610e4fc8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:37.479640 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 177/367] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:37.481091 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55650e-2dd0-4766-9af1-aed26402009f None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.482087 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55650e-2dd0-4766-9af1-aed26402009f None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.482597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55650e-2dd0-4766-9af1-aed26402009f None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.488744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 178/368] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:37.488744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55650e-2dd0-4766-9af1-aed26402009f None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.520320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25237a-731a-4606-9e13-fd59f8b8e372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:37.521502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25237a-731a-4606-9e13-fd59f8b8e372 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.523120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25237a-731a-4606-9e13-fd59f8b8e372 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.523654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25237a-731a-4606-9e13-fd59f8b8e372 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.535592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25237a-731a-4606-9e13-fd59f8b8e372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8e25237a-731a-4606-9e13-fd59f8b8e372 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:37.541416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:37.542455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:37.543515 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:37.546311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 191/369] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:37.548099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.548623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.549268 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.568989 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:37.570685 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:37.607129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55650e-2dd0-4766-9af1-aed26402009f None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:37.618646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:37.650937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:37.652025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.659322 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 192/370] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:37.659322 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.659553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.659943 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.681167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:37.683521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:37.706726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:37.707843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:37.708924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:37.708924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:37.708924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:37.709538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 179/371] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:37.715131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.741838 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:37.850037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba27ae9-374a-4cdc-a1cd-5a15b1d49bee None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:37.917323 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f81b76-c355-4316-b80a-3f169b66ba72 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:37.925507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:37.926479 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.926969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.927501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.928394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:37.928930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:37.930523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:37.951193 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 193/372] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:37.954252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:37.974314 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:37.974937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:37.975226 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:37.975530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:37.976185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:37.976494 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:37.977864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.005605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 180/373] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:38.006548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.032482 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.107717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5679aa-d47c-426a-9602-f75617596ca0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.121550 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.125618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de56fe-e52a-480a-a645-f1e6fa0432cc None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.154387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.156351 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.156976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.157466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.163994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.164632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.164915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.165192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.173764 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.175767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.176527 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:38.181829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:38.182870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 181/374] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:38.188324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:38.198758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:38.200103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:38.201562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:38.201562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:38.201562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:38.201562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 194/375] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:38.201562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.281407 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.301093 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7b474-fed4-4f08-a169-fd5add7757bb None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.311814 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e51dd-1e2f-4a34-b37b-6e5883f38a5b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.338999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.339704 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.339961 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.340243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.346716 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.347340 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.347629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.347752 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.347916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.348563 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:38.348869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.349141 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.350253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.358752 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 195/376] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:38.360261 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-c1a19dc8-5dcc-4142-8e81-3f956ecd46dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:38.363036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:38.364686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 182/377] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d3b96-dcb1-4686-a57c-8577b338c008 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.365993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d3b96-dcb1-4686-a57c-8577b338c008 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.366827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d3b96-dcb1-4686-a57c-8577b338c008 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.367705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 183/378] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:38.371081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d3b96-dcb1-4686-a57c-8577b338c008 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.401625 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.481214 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd6c81-6390-404a-9dd6-61ad33d949b9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.489309 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d3b96-dcb1-4686-a57c-8577b338c008 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.516378 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.516986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.517270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.517546 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.518207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:38.518511 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.519914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.524045 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.524677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.524964 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.525249 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.531357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 196/379] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:38.532140 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.534745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.537493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.539224 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-d47cb754-75f0-4a1c-ad0c-da1939d5b66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:38.554297 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:38.555460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:38.556596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:38.556596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:38.556596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:38.556596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 184/380] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:38.556861 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b464f9-04c9-4061-bd1b-6bb89ca8f13e None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.557079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b464f9-04c9-4061-bd1b-6bb89ca8f13e None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.557363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b464f9-04c9-4061-bd1b-6bb89ca8f13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.574858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-d4b464f9-04c9-4061-bd1b-6bb89ca8f13e None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:38.576103 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-d4b464f9-04c9-4061-bd1b-6bb89ca8f13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:38.618081 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.635726 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45c374-352e-426f-a2d6-0498b0cc84a9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.642432 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 185/381] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:38.644249 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259784dc-9d47-45a0-ae37-80ea2fb0833f None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.644543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259784dc-9d47-45a0-ae37-80ea2fb0833f None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.644826 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259784dc-9d47-45a0-ae37-80ea2fb0833f None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.646283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 186/382] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:38.648215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259784dc-9d47-45a0-ae37-80ea2fb0833f None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.673857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.675017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.675340 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.675624 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.676425 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:38.676749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.678538 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.720412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 197/383] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:38.720412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.768577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259784dc-9d47-45a0-ae37-80ea2fb0833f None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.797634 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.814458 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.816345 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.817085 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.817582 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.827159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdcbb5-0691-424e-af4f-860780ad2e6d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:38.827678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.830945 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-0d6690c5-bf11-48a9-8bc5-b791c3ff3866 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:38.845030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:38.846292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 187/384] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:38.847923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.848693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.848693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.866464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:38.867725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:38.868666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:38.869301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:38.869664 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:38.870016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:38.871041 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:38.871041 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:38.872458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:38.882902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:38.891120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 198/385] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:38.891120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.944446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 188/386] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:38.945283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:38.994231 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffcfe-d801-4444-afd7-743212cdca41 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.029112 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.029823 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.030117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.030454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.042174 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.043661 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:39.058495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:39.059776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:39.061440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:39.061440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:39.061440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:39.061440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 199/387] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:39.061440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.073124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dea482-8bba-4116-ac43-4e2d617b27b1 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.087641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.114596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.115228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.115823 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.116119 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.116889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:39.117209 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.118650 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.119493 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.163287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 189/388] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:40:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:39.163287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.193818 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4b741-3be1-41b9-b0aa-1e9a61c67ef0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.228893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.229514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.229798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.230072 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] PATH_INFO: `/v3/users/glance` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.237488 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.238933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:39.251144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:39.257198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 200/389] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:39.258451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.268401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d147-81df-4948-a6e7-0dc6eb2cb9d9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.303007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.303686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.303977 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.304259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.314280 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:39.319264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:39.320487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 190/390] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:39.321549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.352076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.358670 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8285b26-bf66-4114-8b1a-bff3fdb816af None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.372165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.393583 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.394185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.394474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.394805 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.395497 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:39.395815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.397220 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.409219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 201/391] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:39.410068 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.427911 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2f21-7e3c-4fa9-b7ff-34201c37ad40 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.462866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.463545 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.463836 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.464116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.474654 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.476167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:39.490660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:39.496796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:39.497997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:39.497997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:39.497997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:39.497997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 191/392] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:39.497997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.515038 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0e67-3029-422a-adbe-ef91703998ee None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.548987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.550141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.550432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.550716 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.560688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.562170 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:39.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:39.578219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:39.579227 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:39.579227 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:39.579227 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:39.579451 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.579547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 202/393] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:39.579547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.599705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22173-e6b3-426d-b5e9-de91003af225 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.606148 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.634343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.634885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.635218 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.635528 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.636258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:39.636571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.638021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.654883 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 192/394] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:39.656305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.681090 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bf8a5-772e-479c-a9c9-9e9b34d9e11b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.715291 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.715926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.716219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.716501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.717168 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:39.717483 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.718857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.731683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 203/395] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:39.731683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.756643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b776-afb1-408b-a5d3-87eb9854e44b None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.791683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.792321 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.792608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.793151 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.799972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.801444 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.804384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:39.813926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:39.821887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 193/396] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:39.823143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.823947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:39.836105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe50c-eb99-475b-9d71-497c505b53f8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.870683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.871312 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.871602 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.871882 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.872526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:39.872836 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.874211 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.886673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 204/397] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:39.887453 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.916742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867f47f-31c8-4b91-a034-3cee7f270bca None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:39.950890 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:39.951577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:39.951866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:39.952146 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:39.952799 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:39.953112 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:39.954537 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:39.966183 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 194/398] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:39.966972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:39.985213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c00a26-ee3b-40c7-b7fe-fecb347463af None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.020253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.020911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.021200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.021507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.025501 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.031252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.032728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.039667 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:40.048076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:40.054616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:40.055875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:40.055875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:40.055875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:40.055875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 205/399] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:40.055875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.068041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f892-1245-4083-b199-fcb995ea4026 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.103465 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.104311 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.104596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.105203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.116445 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.117898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:40.132480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:40.133745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:40.134796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:40.134796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:40.134796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:40.134796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 195/400] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:40.134796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.153939 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c27e87-96e3-4732-8dd0-755f6a98bba5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.187202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.187834 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.188123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.188406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.189105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:40.189419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.190913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.208045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 206/401] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:40.208831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.234695 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561bb84-04f4-4659-80e3-1f0e6acc97a1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.248035 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.266834 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.270502 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.271125 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.271456 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.271734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.278329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.279832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:40.291981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:40.293203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 196/402] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:40.294605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.313643 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6538-85c9-4b4b-9a19-9be83ba6225a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.349092 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.349702 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.349984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.350257 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.350896 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.352360 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.394925 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0dfcc-52ea-4de7-a0d9-6f6fbb595eef None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.429096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.429771 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.430134 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.430487 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.431057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 207/403] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:40 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:40.431280 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:40.431679 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.431914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.433186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.452034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 197/404] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:40.452034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.487337 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.552812 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71053fdb-3d75-4fcb-b194-1b3e5a946ca1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.566638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6d720-ecb8-4277-9677-0f93d8372d3e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.567972 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.589734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.590359 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.590677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.590915 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.591613 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:40.591919 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.593312 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.608067 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 208/405] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:40.608270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.614725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.615592 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.616467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.616467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/921c6749f51c49d78748876ec65bd71f/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.636108 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=921c6749f51c49d78748876ec65bd71f, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.637652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.712502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afd53-ec51-4aa8-9af4-bcdeec95a63b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.731330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 198/406] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:40 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/921c6749f51c49d78748876ec65bd71f/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:40.731330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.773728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.774437 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.775016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.775324 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.777100 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:40.777424 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.779066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.799926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 209/407] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:40.801238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.850472 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.855123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:40.879539 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c500-bd7f-4a1e-9906-e45235545e9f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.930021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.931069 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.931601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.932099 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.937126 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d20cbc-6d2b-49b2-bc61-7395d2988703 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:40.943011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.944622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:40.957228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:40.958296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 199/408] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:40.959853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:40.981922 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:40.982490 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:40.982779 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:40.983529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:40.993649 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:40.995442 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:41.009164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:41.010277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:41.011394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 210/409] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:41.015963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:41.086203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea7f48-84a1-4a20-8072-329ef2df91ed None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:41.128946 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:41.141902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee680bd-b4fb-4c51-a8d0-2c7de545dbf2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:41.156077 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:41.156890 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:41.157209 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:41.157499 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/467461dd0dda4187b239000585cf8558/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:41.177196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:41.188213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=467461dd0dda4187b239000585cf8558, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:41.189931 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:41.211135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:41.211816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:41.212115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:41.212906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:41.213626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:41.213950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:41.216199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:41.230360 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 211/410] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:41.231424 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:41.306253 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 200/411] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:40 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/467461dd0dda4187b239000585cf8558/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:41.307134 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:41.453580 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33db6d6-555f-404c-8447-4d45ec4a603c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:41.581629 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5bcdf-a1c6-4afc-ad51-890da8766782 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:41.583521 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:41.584609 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:41.585328 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:41.585328 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:41.588806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:41.588806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:41.588806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:41.626729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 212/412] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Oct 17 05:40:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:41.630961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:41.656632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:41.661264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:41.661908 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:41.662189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:41.662466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/168a505da7f144acaabd97443ed5928e/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:41.692745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=168a505da7f144acaabd97443ed5928e, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:41.694201 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:41.757483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 201/413] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:41 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/168a505da7f144acaabd97443ed5928e/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:41.783445 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d35f31-ad0e-497a-a4c3-d79ad67d78a9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:41.824257 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:41.824899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:41.825185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:41.825463 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:41.842378 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:41.843979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:41.861447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:41.862677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:41.863723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:41.863723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:41.863723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:41.863723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 213/414] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:41.868598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:41.940717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:42.005351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa08f8-bad4-49d5-9a77-04e32cf459d5 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:42.042936 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:42.043594 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:42.044021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:42.044300 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:42.045009 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:42.045322 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:42.046943 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:42.059192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 202/415] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:42.065887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:42.147110 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:42.187885 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c69c4e-d9d7-4295-ace9-f669330e724e None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:42.223945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:42.224588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:42.224927 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:42.225213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:42.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:42.233848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:42.247210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:42.249435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:42.251498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 214/416] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:42.256304 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:42.369041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdbac8-aed8-48a3-99da-8005e6afeace None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:42.371677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:42.405345 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:42.405882 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:42.406168 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:42.406516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:42.407225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:42.407570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:42.409062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdae39-91a1-4176-8152-cccdfbd778bf None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:42.428514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 203/417] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:42.436590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:42.553238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d714cc7-7e33-407a-8d68-0215c1d7db1c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:42.578966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:42.591798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f509b-2d78-437c-a1c7-7f261f8487ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:42.592458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f509b-2d78-437c-a1c7-7f261f8487ec None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:42.592787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f509b-2d78-437c-a1c7-7f261f8487ec None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:42.593070 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f509b-2d78-437c-a1c7-7f261f8487ec None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/6f280acb87494dc59f0f7c222edcc87e/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:42.609853 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f509b-2d78-437c-a1c7-7f261f8487ec None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=6f280acb87494dc59f0f7c222edcc87e, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:42.611522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f509b-2d78-437c-a1c7-7f261f8487ec None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:42.684051 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 215/418] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:42 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/6f280acb87494dc59f0f7c222edcc87e/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:43.058892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2516-4e90-4a01-9e22-337d0a275bc0 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:43.059479 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2516-4e90-4a01-9e22-337d0a275bc0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:43.059479 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2516-4e90-4a01-9e22-337d0a275bc0 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:43.061095 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 204/419] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:43.066754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ca34-046f-4c55-a089-2da3f4e61f1d None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:43.067186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ca34-046f-4c55-a089-2da3f4e61f1d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:43.067607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ca34-046f-4c55-a089-2da3f4e61f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:43.087502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-67a4ca34-046f-4c55-a089-2da3f4e61f1d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:43.088833 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-67a4ca34-046f-4c55-a089-2da3f4e61f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:43.197899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 216/420] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:43.302333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733afb2-05cf-49b0-aa11-5a81f94eea31 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:43.302627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733afb2-05cf-49b0-aa11-5a81f94eea31 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:43.302915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733afb2-05cf-49b0-aa11-5a81f94eea31 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:43.304595 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 205/421] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:43.309967 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:43.310435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:43.310891 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:43.340283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:43.341728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:43.424863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 217/422] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:43.718913 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733afb2-05cf-49b0-aa11-5a81f94eea31 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:43.852021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733afb2-05cf-49b0-aa11-5a81f94eea31 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:43.860658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:43.903104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:43.935275 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:43.935965 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:43.936250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:43.936531 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:43.964395 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:43.975089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:43.976170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:43.977131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:43.981007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 206/423] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:43.992354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.023230 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9cb85-8f70-4b39-91c6-69b1cbc464e8 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.025995 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:44.064163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4368bd3-a814-44fa-ab57-60b2841852e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.065350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4368bd3-a814-44fa-ab57-60b2841852e1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.065646 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4368bd3-a814-44fa-ab57-60b2841852e1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.065947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4368bd3-a814-44fa-ab57-60b2841852e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.076662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4368bd3-a814-44fa-ab57-60b2841852e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c4368bd3-a814-44fa-ab57-60b2841852e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:44.081589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:44.082662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:44.083694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 218/424] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:44.084511 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad8a6-bbb1-4b7a-939f-87b46a4732b3 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.084511 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad8a6-bbb1-4b7a-939f-87b46a4732b3 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.085162 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad8a6-bbb1-4b7a-939f-87b46a4732b3 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.090266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 219/425] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:44.095270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caad8a6-bbb1-4b7a-939f-87b46a4732b3 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.123797 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92bec7-2c5e-4d3f-8289-f8e2e25ecfdb None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.200221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:44.203778 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.205286 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.205928 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.206436 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.207334 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] RBAC: Adding query filter params (name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:44.207716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.211065 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caad8a6-bbb1-4b7a-939f-87b46a4732b3 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.211196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225eaa1-3d66-4cd4-b295-ac2db9df91ec None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.221656 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 207/426] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:44.223071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.223396 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.223687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.247622 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.248587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.249088 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.249630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.249888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:44.250808 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:44.251109 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:44.251629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.253242 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.262747 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 220/427] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:44.263928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.355597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 208/428] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:44.357175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.386788 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c90d69-ae15-4340-9b1c-dfa83e4468ca None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.397197 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:44.434449 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.436580 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.437769 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.438884 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.461909 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.464984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fbe9a427-1ede-40de-b5f4-633ff45e5908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:44.487986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:44.489058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:44.490111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:44.490111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:44.490111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:44.491916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 221/429] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:44.493280 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09872569-1b4a-4f12-be9f-ac73ea2936c5 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.493795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09872569-1b4a-4f12-be9f-ac73ea2936c5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.494278 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09872569-1b4a-4f12-be9f-ac73ea2936c5 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.495972 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 222/430] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:44.498002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09872569-1b4a-4f12-be9f-ac73ea2936c5 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.511736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:44.545822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e57db-4e16-44e3-85f7-c04ef52950b0 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.594657 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.595328 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.595611 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.595923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.609966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.612263 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.612715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09872569-1b4a-4f12-be9f-ac73ea2936c5 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-0ac47e29-8039-409e-8108-f321168153b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:44.631622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:44.632745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:44.633743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:44.633743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:44.633743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:44.633743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 209/431] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:44.634008 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.634591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.634591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.652491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.653524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.654050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.654574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.655504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:44.656138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:44.656352 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.657343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:44.658308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.675693 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 223/432] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:44.676915 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.725717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:44.733687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 210/433] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:44.737329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.751646 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:44.795065 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03c02c-243e-4668-871f-f04869552c03 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.837795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.838400 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.838689 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.838971 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.839657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:44.839968 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.841357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.854037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 224/434] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:44.854809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:44.883769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd79062-f03d-4066-bef1-a905d39973f9 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.934427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:44.935821 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:44.936115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:44.936395 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:44.950213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:44.952071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:44.966039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d000d8b-4a26-4cf3-b07a-17f6d2f67693 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:44.970802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:44.971903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:40:44.972981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:44.980238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 211/435] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:44.980414 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.000375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.013865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.013865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.013865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.013865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.023544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:45.028599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:45.029951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:45.031104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:45.035958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 225/436] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:45.037408 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.054052 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.103117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b52912-2db4-4946-bef4-6d2fe2988fb8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.171983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445aad8-5369-4c9c-90a3-c4e6efb73618 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.176119 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.176806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.177086 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.177362 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] PATH_INFO: `/v3/users/nova` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.188360 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.190397 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:45.203374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:45.204627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 212/437] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:40:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:45.206194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.215887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.216724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.217167 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.217603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.218911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:45.219439 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.222365 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.241194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 226/438] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:45.242277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.293346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.317767 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b8365-5475-47c6-b529-61a992ac2318 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.334145 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.346471 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b554e-7507-4f79-b7c8-22a49b03bfad None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.352414 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.353065 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.353351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.353628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.364411 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:45.369605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:45.370824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 213/439] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:45.371855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.384256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.384868 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.385154 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.385436 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.386108 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:45.386418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.387853 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.396618 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 227/440] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:45.397258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.472827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849418-a200-4613-9429-9bc24b9260f3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.497169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd358fe-c751-490e-a404-2430f5307c99 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.506380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.506968 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.507092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.507290 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.507603 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.508324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:45.508641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.510095 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc4d6-a523-484a-9841-b0679f8ee581 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.527363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 214/441] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:45.528571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733c67e-3ca3-47ad-bdfe-0ce06e103bf0 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.528915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733c67e-3ca3-47ad-bdfe-0ce06e103bf0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.529142 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733c67e-3ca3-47ad-bdfe-0ce06e103bf0 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.530358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 215/442] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:45.531048 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733c67e-3ca3-47ad-bdfe-0ce06e103bf0 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.534166 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.534887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.535180 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.535492 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.546342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.548018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:45.562748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:45.563795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:45.565316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:45.565316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:45.565316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:45.565316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 228/443] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:45.565316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.571262 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.637389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733c67e-3ca3-47ad-bdfe-0ce06e103bf0 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.664839 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cda42-9509-4113-a5f7-edd7c9c764c2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.671009 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.671917 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.672201 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.672480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.673141 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] RBAC: Adding query filter params (name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:45.673442 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.674910 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.682898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 216/444] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:45.683640 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.701772 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.702408 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.702696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.702978 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.714098 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.715585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-683180e7-801a-4ef7-8ff2-7e9256d78276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:45.729626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:45.730736 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:45.731937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:45.731937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:45.731937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:45.731937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 229/445] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:45.731937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.732256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.732410 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.749880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:45.751076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:45.758956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.784703 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:45.790566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc237d-2b33-4acb-9410-5f1ddfa74624 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.819691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 230/446] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:45.820492 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.835154 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.835614 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.835962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.836227 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.914356 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.915866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.922106 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d258cb-f5df-4e37-9f42-1ccd3de9d808 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:45.958186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:45.958806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:45.959135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:45.959465 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:45.960124 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:45.960435 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:45.961816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:45.978021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 231/447] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:45.978021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:45.978021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.024260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.031025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 217/448] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:46.031778 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.090500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63860460-ed74-489b-8b1c-cff80ac6e8a9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.117274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.117903 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.118183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.125023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.129484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.131028 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.138021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9ee15-92fa-4cae-a39a-5e7e8df83ea0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:46.145713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:46.146966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:46.148046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:46.148046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:46.148046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:46.148046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 232/449] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:46.148046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.199165 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.199804 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.200089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.200371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.201014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:46.201323 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.202706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.213922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 218/450] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:46.214711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.221808 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.267393 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609f8a9-e073-47ef-92ac-5983202009c9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.270619 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.301547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.302172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.302459 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.302740 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.309458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.310899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.315760 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ebfb4-3622-48cf-9522-338e4f4a8d7c None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:46.322477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:46.323489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:46.324512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 233/451] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:46.325069 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.353830 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.354390 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.354672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.354951 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.365631 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.367092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:46.381525 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:46.382600 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:46.384023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:46.384023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:46.384023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:46.384023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 219/452] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:46.384023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.426316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce83270-28ce-4586-a4e4-d67362682cd1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.440007 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.459944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.460568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.460854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.461134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.472218 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:46.477387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:46.478704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 234/453] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:46.479924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.487655 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c30f26-cff9-4050-99b8-1f88d9283c61 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.496604 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.520674 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.521282 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.521569 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.521842 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.522486 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:46.522795 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.524203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.536915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 220/454] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:46.537895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.583163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5636533-15af-4f25-9a4c-516fdec3ae98 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.615907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.616456 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.616739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.617016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.624404 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.625865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.635502 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fecc11-7083-4b8b-8621-361f07e383e4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:46.636855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:46.637996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 235/455] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:46.639052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.673413 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.675186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.675825 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.676108 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.676384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.677072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:46.677381 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.678811 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.690334 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 221/456] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:46.691118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.724190 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.740749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5c913-147b-4e23-ad5e-cc7b51acb9a5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.774535 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.775136 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.775446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.775723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.776370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:46.776675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.778031 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.789541 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 236/457] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:46.790585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.797565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1558001-361d-4434-a0ef-46c707981f77 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.834701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.835461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.835750 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.836027 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.836683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:46.836989 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.838374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.846176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 222/458] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:46.846953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.889577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.895830 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11182d91-f959-41ff-8ed1-770d9f26d183 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.929669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.930298 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.930586 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.930866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.937652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:46.938177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.940229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:46.947530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12b77-9cdd-4341-98f1-d672a7115bf7 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:46.951729 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:46.958809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 237/459] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:46.959988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:46.983853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:46.984840 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:46.985923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:46.986230 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:46.987540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:46.987857 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:46.989298 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.006596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 223/460] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:40:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:47.007424 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.058045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73db7b9-78eb-4cfd-8a11-6797eac653d2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.092770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.093414 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.093695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.093979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/6639d3279dfc42dba4073a200a09a357/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.107956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=6639d3279dfc42dba4073a200a09a357, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.108496 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670814e-b36a-440d-a692-e9480ff2c8e3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.109405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.142341 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.143581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.143868 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.144150 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.150765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.152231 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:47.163660 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:47.164643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 224/461] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:47.165652 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.175999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.225017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 238/462] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:46 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/6639d3279dfc42dba4073a200a09a357/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:47.225848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.229338 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.286951 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3298d-0945-47fd-8d3a-f60d327d06a4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.328807 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.331496 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.331799 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.332216 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.340253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3110c-063d-431e-8371-8738ab5e9829 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.345483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.347010 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:47.366431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:47.367636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:47.368646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:47.368646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:47.368646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:47.368646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 225/463] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:47.368646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.382991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.383646 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.383933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.384208 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.385357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:47.385357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.386712 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.407663 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 239/464] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:47.408487 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.449141 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.472985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.496314 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4827b-2dd7-4f24-a997-55fbc4f5fdd6 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.513284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6af87-773e-4950-a2e1-0e0c9075ed36 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.531936 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.532556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.532844 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.533688 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.551281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.551944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.552211 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.552488 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.553195 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:47.553507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.554945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.567201 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 240/465] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:47.568043 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.614045 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.615548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.680983 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.683349 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55d376-8f83-47c4-ae11-329ce393b0b2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.706994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.722684 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.724002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.724579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.725067 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.725927 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:47.726475 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.727919 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 226/466] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:47.731471 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.733268 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.742334 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 241/467] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:47.743078 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.847603 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e958b9-5d4e-4048-a2ee-7cf2bc6c99bd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.851542 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd1fd4-4396-482a-9750-f00c3278be56 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:47.884434 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.885050 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.885338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.885617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.889076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:47.889684 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:47.889966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:47.890244 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:47.890965 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=921c6749f51c49d78748876ec65bd71f) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:47.891276 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.893282 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:47.911334 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 242/468] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/role_assignments?user.id=921c6749f51c49d78748876ec65bd71f&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:47.911334 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:47.932631 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.977238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:47.984962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:47.986809 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.042076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98152c-81f3-4399-ac89-d35d41aec9f1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.082508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.083461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.083816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.087618 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.097529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.099585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.116461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 227/469] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:48.117340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:48.117996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:48.120933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:48.121925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:48.121925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:48.121925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:48.121925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 243/470] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:48.128111 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.192729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:48.241932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603772a-7af2-47a5-b038-0e75b09dce2b None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.243230 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d18e5-eeb5-4f72-b026-22ca64f4d70d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.251661 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:48.285643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.286895 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.287345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.287617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.288085 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.288412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.288763 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.288875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.289492 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:48.289866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.291702 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.299392 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.300843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.308029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 244/471] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:48.309143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:48.318004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:48.319073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 05:40:48.320179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:48.323757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 228/472] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:48.324608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.425815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4b4a0-5874-4fce-834e-63e67d03981c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.454561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de7472-2fa0-4262-ba4a-dd6420081349 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.466223 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.467022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.467321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.467983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.478001 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:48.481300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.483599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.491794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.492938 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.499994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.499994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:48.501156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:48.502707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:48.503753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:48.503753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:48.503753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:48.503963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.504043 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 245/473] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:48.504043 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.508387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.523683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:48.525763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:48.526729 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:48.527818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:48.534934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 229/474] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:48.536079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.619532 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f065-78e4-4d41-a6b5-6fc7c8faebd8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.656015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.656943 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.657481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.658043 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.659166 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:48.659755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.661519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.665765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6beae-5e08-4c9a-af5e-22add803a24a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.686090 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 246/475] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:48.687312 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.706743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.707283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.707655 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.707940 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.708751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:48.709065 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.710744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.730561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 230/476] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:48.730561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.769420 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:48.780385 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:48.811611 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c068b7-9b3c-41a4-8e9c-423d48b0add3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.850560 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.851640 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.852238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.852751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.853286 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418b6c-f971-46d0-90c4-52499b4d983b None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:48.853918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:48.854442 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.856314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:48.874717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 247/477] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:48.875920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:48.895453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:48.896104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:48.896517 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:48.896796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:48.983176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:48.985221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.000921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855734-2308-440f-aadb-68ea36ae5324 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.032080 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:49.037440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:49.047397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.048042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.048321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.048596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.058167 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.059717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:49.074549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:49.080577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 248/478] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:49.081656 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.112130 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 231/479] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:49.113710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.210145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa958-5ec3-443c-905b-83b7a5a5f74e None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.231046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb5a1b-deaa-4d6a-ac4b-56b6f44fd0de None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.257217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.258217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.258717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.259210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.260200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=467461dd0dda4187b239000585cf8558) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:49.260755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.262415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.269426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 249/480] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/role_assignments?user.id=467461dd0dda4187b239000585cf8558&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:49.275512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.275512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.275512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.275512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.275935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:49.275935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.276538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.277480 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.288143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:49.297896 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 232/481] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:49.299871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.427029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:49.444281 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c751e2d-3558-44d2-991c-24b821a0ddd2 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.485410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.486509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.486956 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.487427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.488438 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=168a505da7f144acaabd97443ed5928e) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:49.489150 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.490852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a80a3-22e7-41c6-acaf-83761715093c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.492316 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc70d-ef84-4be6-b7cb-d909ab48e73c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.505289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 233/482] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/role_assignments?user.id=168a505da7f144acaabd97443ed5928e&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:49.507322 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9920-6375-4db3-b585-16767c5e4997 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.507776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9920-6375-4db3-b585-16767c5e4997 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.508339 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9920-6375-4db3-b585-16767c5e4997 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.510214 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 234/483] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:49.517209 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.518301 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.518585 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.540722 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.541388 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.541669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.542026 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.550178 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:49.551907 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:49.555323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.557804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:49.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:49.577815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:49.578837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:49.578837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:49.578837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:49.578837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 250/484] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:49.583343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.633927 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:49.645245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 235/485] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:49.723455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee69cf-7e40-4a5e-b9ba-d64b98ef6fc6 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.729430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:49.764712 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.765247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.765602 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.765889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.766646 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:49.767889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.769617 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.783097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 251/486] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:49.788300 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.934409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72918827-fcfe-4263-9fab-f4e39f883dfc None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:49.968711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:49.969353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:49.969636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:49.969911 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:49.974988 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:49.977543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:49.979002 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:49.990806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:49.991855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:49.992869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 236/487] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:49.998120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:50.088618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:50.146073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133632-9a1b-4614-88ac-7b18aee40be7 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:50.168509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fe11b-0051-489c-8fe1-029509be2f48 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:50.218117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:50.219159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.219690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.220219 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.221698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:50.223389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:50.226847 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:50.240348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:50.241750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.242246 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.242748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.253878 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 237/488] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:50.259814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:50.264447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:50.274985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:50.276102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:50.277057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:50.279804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 252/489] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:50.285766 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:50.333998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:50.410686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a3fd-3adb-4421-8d0c-966e67a86329 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:50.428950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:50.429725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e292-2ee4-42be-b356-2f79e7b25035 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:50.459655 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:50.460952 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.461642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.462076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.463231 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=6f280acb87494dc59f0f7c222edcc87e) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:50.464123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:50.466164 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b842c-43a1-44cb-8abe-0d5d6ad4b0e2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:50.468732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:50.469469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.469853 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.470484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.471659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:50.472108 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:50.473591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:50.479548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 238/490] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:50 2022] GET /identity/v3/role_assignments?user.id=6f280acb87494dc59f0f7c222edcc87e&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:50.479548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14730e0-c15d-4f89-8d98-e7c88cc032f6 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.479548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14730e0-c15d-4f89-8d98-e7c88cc032f6 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.479548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14730e0-c15d-4f89-8d98-e7c88cc032f6 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.479548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 239/491] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:50.496583 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.496942 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.497145 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.505045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 253/492] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:50.512848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:50.543221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:50.544560 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:50.639345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:50.656737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c935207-78ee-4712-85d9-a1e8c34e20dd None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:50.671198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 240/493] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:50.719206 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:50.719993 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.720287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.721174 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.738026 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:50.739899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:50.761096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:50.762150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:50.763181 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:50.763181 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:50.763181 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:50.765310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 254/494] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:50.770755 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:50.877449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0414f-9b6a-4296-9e4d-20cf2b7d78fd None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:50.887014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:50.913540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:50.914161 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:50.914446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:50.914725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:50.915400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:50.915708 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:50.917084 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:50.929610 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 241/495] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:50.934720 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.041519 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.053954 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b690-537e-4dc4-833b-6299bfaddcf3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.103811 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:51.106854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.107196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.107508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.122317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:51.123993 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:51.125969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:51.140606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:51.141683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:40:51.143761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:51.149814 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 255/496] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:51.150804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.198360 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d072c-0644-431d-8d89-95134d0ebeba None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.234555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:51.235199 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.235516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.235796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.247187 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:51.253207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:51.254241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:51.255184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 242/497] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:51.264262 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.293992 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:51.370187 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06ee83-3d6a-4c44-ade4-24086d3571be None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.405143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:51.405766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.406054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.406334 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.407202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:51.407704 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:51.409730 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:51.413171 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631085e-fc4a-4627-8d3b-6337ff8844b7 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.418801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 243/498] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:51.425402 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.483894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:51.484433 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.484719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.484996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.486451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:51.486971 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:51.488457 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:51.530671 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 256/499] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:51.538189 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.611671 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:51.625203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a396d-9d7f-458c-8182-aed4734ff353 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.653750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:51.677472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:51.678272 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.678618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.678910 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.697814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:51.700610 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:51.733584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:51.734769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:51.735985 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:51.735985 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:51.735985 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:51.740315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 244/500] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:51.742439 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b033697-70d3-45ef-978b-9bae401c72d5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.748904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:51.799320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa57b-b3ae-4435-8ebb-42f1f307d65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:51.799320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa57b-b3ae-4435-8ebb-42f1f307d65e None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.799320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa57b-b3ae-4435-8ebb-42f1f307d65e None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.799320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa57b-b3ae-4435-8ebb-42f1f307d65e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.879029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa57b-b3ae-4435-8ebb-42f1f307d65e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:51.880594 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa57b-b3ae-4435-8ebb-42f1f307d65e None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:51.940045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:51.953416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd309e6-8a62-415a-84fd-3d70f2785707 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:51.987641 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 257/501] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 05:40:51.988934 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2864232-224c-449f-93b4-29e3fa3b506c None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:51.989224 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2864232-224c-449f-93b4-29e3fa3b506c None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:51.989503 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2864232-224c-449f-93b4-29e3fa3b506c None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:51.991056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 258/502] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:51.993067 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2864232-224c-449f-93b4-29e3fa3b506c None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.003915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.004538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.004824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.005107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.005762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:52.006081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.007489 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bbf0e-06ed-467b-a8c8-9896d3fb0cd2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.020841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 245/503] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:52.023831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.024120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.024394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.039637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.047974 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:52.049843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:52.111678 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2864232-224c-449f-93b4-29e3fa3b506c None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.137653 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 246/504] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:52.138594 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.168151 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.168929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.169252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.169495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.193591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.195083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:52.224635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:52.225768 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:52.226785 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:52.226785 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:52.226785 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:52.237409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 259/505] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:40:52.242590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.254721 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e336ff-3796-431c-b943-54a100ea78e3 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.272608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.293852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.298880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.298880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.298880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.305276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.307563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-9cbe497d-6ff8-4c4c-9dd4-d9fff8e65cb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:52.322581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:52.323779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 247/506] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:52.324884 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa16a2-64c8-4bee-b670-535f96de11d0 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.325635 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa16a2-64c8-4bee-b670-535f96de11d0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.325635 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa16a2-64c8-4bee-b670-535f96de11d0 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.330680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 248/507] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:52.330680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.332853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.333144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.333427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.353005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:52.354490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:52.400105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15587a-2d44-416a-99c4-51c43418b681 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.435451 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 249/508] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:52.436333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.450270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.450995 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.451313 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.451640 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.452299 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:52.452607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.454095 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.467646 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 260/509] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:40:52.468573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.550182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.550489 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c25b-aff2-4ac0-adc3-3ae1843dbfae None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.580996 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.596264 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0103e-5703-48a0-8af3-252ad07c7307 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.600589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.601453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.601946 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.602462 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.603561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:52.604688 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.606784 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.633783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 250/510] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:52.634942 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.639440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.639788 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.640150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.640503 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.641170 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.643164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:52.655183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:52.656429 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:52.657397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 261/511] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:52.658388 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.774757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525940da-7ad5-4033-b425-911c2a892ee0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.785287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c73a5-085e-47e1-948a-54b73b9dcb43 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.816663 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.817244 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.817548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.817811 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.818741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.823952 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.824588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.824876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.824982 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.825236 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.826374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.837413 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:52.838642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:52.839646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:52.840710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:52.841207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.841286 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 251/512] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:52.841286 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:52.857163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:52.858290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:52.859326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:52.859326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:52.859326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:52.862908 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 262/513] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:52.863771 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:52.889167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:52.950763 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84cff-897c-43a4-bcd1-f8de0fdc0642 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.969784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdad0-c6fe-4c10-a2be-9b4e656472a4 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:52.989962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:52.990607 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:52.990894 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:52.991176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:52.991871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:52.992190 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:52.993693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:52.998926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 252/514] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.000330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.009930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.010582 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.010869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.011150 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.022767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.024253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:53.039430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:53.040523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:53.041529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:53.041529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:53.041529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:53.045556 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 263/515] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:53.046409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.082740 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:53.115567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d875d88-3042-4ce0-b833-49b17015edb8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.117898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:53.148228 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b2b6c-8199-46d0-9181-65e4a617735c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.150025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.150637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.150930 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.151207 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.151935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:53.152247 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.153683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.166966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 253/516] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.168236 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.187508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.188333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.188618 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.188895 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.190059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:53.190363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.191774 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.203799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 264/517] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.204675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.273471 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e9764-0c81-44c5-a95a-f0af0ef943f9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.295528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:53.301428 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f693be-509b-4b65-bc5c-5ffb8aee579c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.307213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.308010 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.308295 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.308572 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.309191 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:53.309499 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.310866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cd061-7487-4539-a2e1-4f6fc09e7428 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.316936 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 254/518] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.318258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e067b-7f9d-4e23-a065-f229ff0aa1ac None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.318546 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e067b-7f9d-4e23-a065-f229ff0aa1ac None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.318954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e067b-7f9d-4e23-a065-f229ff0aa1ac None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.320204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 255/519] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:53.320933 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e067b-7f9d-4e23-a065-f229ff0aa1ac None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.355913 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:53.361692 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.366310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.366310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.366310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.366310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:53.366310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.366310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.379164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 265/520] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.380097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.452276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e067b-7f9d-4e23-a065-f229ff0aa1ac None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.491398 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.492015 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.492302 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.492587 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.493323 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=6639d3279dfc42dba4073a200a09a357) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:53.493583 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2451-1cef-48b4-895b-7b0e623bd8c2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.494028 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.495074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded72c5c-0d90-49cb-976a-e54a873fa8f1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.504136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 256/521] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/role_assignments?user.id=6639d3279dfc42dba4073a200a09a357&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.505439 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.506507 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.506688 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.532304 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:53.532792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:53.540466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.541406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.541848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.542263 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.552566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.555128 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:53.573914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:53.575182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:53.576421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:53.578738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 266/522] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:53.580595 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.609371 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:53.618427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 257/523] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:53.625688 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.677280 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:53.714831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9df07-1f4a-4b49-8533-6604fdc68a30 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.762607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.763226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.763561 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.764084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.764502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.765945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.791599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632f9fd-c680-47b9-8dfc-fb34b4fee524 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:53.834091 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:53.834804 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:53.835135 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:53.839350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:53.839350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:53.839350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:53.839350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:53.852570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 258/524] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:53.858252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.864398 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 267/525] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:40:53 2022] POST /identity/v3/users => generated 254 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:53.943733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:53.967175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:54.009126 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912fcb8-8049-4e72-81d2-14c33073a22c None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:54.073206 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:54.080417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:54.081159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:54.081449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:54.081735 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/921c6749f51c49d78748876ec65bd71f/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:54.087114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=921c6749f51c49d78748876ec65bd71f, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:54.088639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:54.121668 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d5b85-6e8e-42a4-ae05-4c4757b51817 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:54.185643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 259/526] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:53 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/921c6749f51c49d78748876ec65bd71f/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:40:54.225743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:54.226466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:54.226804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:54.227092 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] PATH_INFO: `/v3/services/network` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:54.227810 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:54.229201 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:54.239870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:54.241165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:54.242751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 268/527] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:53 2022] GET /identity/v3/services/network => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:54.248495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:54.391860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092343-4abb-439c-acaf-1b325b6feb9e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:54.441581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:54.442211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:54.442496 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:54.442775 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:54.443504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] RBAC: Adding query filter params (name=network) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:54.443820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:54.445253 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e8733-7012-4800-9660-60f8388b50ee None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:54.450878 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 260/528] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:40:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:54.456864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:54.565954 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac888a-fca9-4d67-bd8d-98d7ef8ff767 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:54.610526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:54.611664 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:54.612193 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:54.612677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:54.613517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] RBAC: Adding query filter params (type=network) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:54.614049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:54.615654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b9610-24f5-4204-8e36-b020864898af None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:54.625196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:54.629786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 269/529] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:40:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:55.886263 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7ad15-1534-4521-a3d8-68e27d178a5a None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:55.886793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7ad15-1534-4521-a3d8-68e27d178a5a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:55.886876 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7ad15-1534-4521-a3d8-68e27d178a5a None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:55.888455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 261/530] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:55.894410 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060688-209f-4f5a-9ba1-9b8fe19d2c64 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:55.894848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060688-209f-4f5a-9ba1-9b8fe19d2c64 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:55.894960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060688-209f-4f5a-9ba1-9b8fe19d2c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:55.924330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-e9060688-209f-4f5a-9ba1-9b8fe19d2c64 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:55.925550 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e9060688-209f-4f5a-9ba1-9b8fe19d2c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:56.025923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 270/531] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:56.391246 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21fc0-39e7-40c1-9ff4-8eae9b5a0251 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:56.391682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21fc0-39e7-40c1-9ff4-8eae9b5a0251 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:56.391866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21fc0-39e7-40c1-9ff4-8eae9b5a0251 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:56.393456 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 262/532] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:56.398850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1e9c-8b33-427c-b43e-4b35ff9645a2 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:56.399147 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1e9c-8b33-427c-b43e-4b35ff9645a2 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:56.399459 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1e9c-8b33-427c-b43e-4b35ff9645a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:56.436303 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21fc0-39e7-40c1-9ff4-8eae9b5a0251 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:56.439097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-ccdf1e9c-8b33-427c-b43e-4b35ff9645a2 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:56.440287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-ccdf1e9c-8b33-427c-b43e-4b35ff9645a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:56.446353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:56.528037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 271/533] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:40:56.529337 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b921-1c59-4cad-b61d-a55e9652d402 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:56.529632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b921-1c59-4cad-b61d-a55e9652d402 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:56.529919 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b921-1c59-4cad-b61d-a55e9652d402 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:56.531425 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 272/534] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:56.537413 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:56.537982 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:56.538271 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:56.568619 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:56.569810 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:56.601071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21fc0-39e7-40c1-9ff4-8eae9b5a0251 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:56.664906 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:56.665441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:56.665726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:56.666001 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:56.666659 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:56.668068 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:56.680495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:56.692030 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 273/535] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:56.788867 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 263/536] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:40:56 2022] POST /identity/v3/services => generated 251 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:40:56.869016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.013556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.034927 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59e8ff-8271-4edc-85f5-40b8168696c3 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.091552 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.116364 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29972c-8dd2-43bd-8032-6999a564dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.117021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29972c-8dd2-43bd-8032-6999a564dca7 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.117309 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29972c-8dd2-43bd-8032-6999a564dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.117588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29972c-8dd2-43bd-8032-6999a564dca7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.130531 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29972c-8dd2-43bd-8032-6999a564dca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:57.133864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e9a5-c8e7-4dad-8e94-6a592ae94010 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-7d29972c-8dd2-43bd-8032-6999a564dca7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:57.135557 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:57.136597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:57.137569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 274/537] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:57.140901 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085e8cb-a04f-4431-b886-7bbe6af28dc2 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.140901 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085e8cb-a04f-4431-b886-7bbe6af28dc2 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.140901 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085e8cb-a04f-4431-b886-7bbe6af28dc2 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.143631 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 275/538] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:57.148128 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.148426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.149081 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.173085 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:57.174321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:57.189528 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.190553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.190997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.191337 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.207805 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:57.216129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:57.217289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 264/539] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:57.218248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.238183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.269606 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 276/540] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:57.271320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.365844 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b9b4e-5f35-45b6-bf84-81006c3c5169 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.387702 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a18b6-233e-45ce-ac9d-253a59c2f139 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.395107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.406013 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.407486 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.407486 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.407486 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.409287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:57.409601 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:57.410980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31ee90-9053-470c-bbd4-228734c1ef29 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:57.427387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 265/541] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:57.429034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bce42f-b5c0-4438-9fa8-0ab3ac90c3aa None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.429399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bce42f-b5c0-4438-9fa8-0ab3ac90c3aa None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.429833 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bce42f-b5c0-4438-9fa8-0ab3ac90c3aa None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.430433 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.431048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.431356 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.431537 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 266/542] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:57.432248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bce42f-b5c0-4438-9fa8-0ab3ac90c3aa None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.436021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.439504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:57.439504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:57.439504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07e04f-29ee-4eef-bb8a-103efdee46bf None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:57.448834 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 277/543] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:57.450237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.450530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.450811 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.469849 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:57.471681 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:57.511500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.554388 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 278/544] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:40:57.555181 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.561043 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bce42f-b5c0-4438-9fa8-0ab3ac90c3aa None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.604783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488749d-9372-454c-9297-c00189654bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.605424 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488749d-9372-454c-9297-c00189654bbd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.605709 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488749d-9372-454c-9297-c00189654bbd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.605988 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488749d-9372-454c-9297-c00189654bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.620327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488749d-9372-454c-9297-c00189654bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:57.621796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488749d-9372-454c-9297-c00189654bbd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-6488749d-9372-454c-9297-c00189654bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:57.644000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:57.645104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:57.646133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:57.646133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:57.646133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:57.646133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 267/545] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:57.646397 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488749d-9372-454c-9297-c00189654bbd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.690055 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb2593-71ed-4877-b702-682ddd443b89 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.697023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.736693 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.737345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.737641 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.737915 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.752222 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:57.754417 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:57.763537 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488749d-9372-454c-9297-c00189654bbd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:57.774522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:57.775738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:57.776835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:57.776835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:57.776835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:57.776835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 279/546] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:57.777437 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.815928 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.818868 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.819551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.819841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.820175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.832339 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:40:57.838225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:57.839361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 268/547] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:57.840369 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.910664 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430baad-669b-41f0-9e53-b8cb50703cfb None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.946467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:57.947579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.948197 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.948481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.948756 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.949283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636499d9-79fe-4f32-ac09-8315947e030a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:57.949400 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:57.949707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:57.951076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:57.968661 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 280/548] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:57.969505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:57.984108 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:57.984728 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:57.985013 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:57.985288 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:57.986216 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:57.986529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:57.987955 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.001247 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 269/549] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:58.002172 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.059441 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.078528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850c4e1-28d8-4816-999e-917a2d930b68 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.114615 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.115266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.115597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.115841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.116172 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791794-f285-42ae-bfc5-ff07e05bae66 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.116614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.117962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.151567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.152190 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.152467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.152746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.153407 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:58.153722 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.155081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.163327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 270/550] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:58.164104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.174368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 281/551] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:40:57 2022] POST /identity/v3/services => generated 240 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:40:58.175262 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.201894 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.318036 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c36fe2-d394-4c2d-969e-4639bf332a40 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.318903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637820b5-6264-40c4-8d8f-c698da49be24 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.361368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.362016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.362301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.362573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.367044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.371074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.371935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.372227 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.372507 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.373592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.375157 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.383288 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.384762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:58.390841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:58.391986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:58.393647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 282/552] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:58.394341 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:58.403503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:58.405949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 271/553] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:58.408669 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.513938 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3302c-7c8e-4cc3-97ae-b28df2cf01e8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.525891 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.552346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.553115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.553496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.553848 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53223c-7333-47cd-9d73-6d9b58f786ff None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.553976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.566758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.568481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:58.584146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:58.585474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:58.586540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:58.586540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:58.586540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:58.591317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 283/554] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:58.592218 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.600136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.601038 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.601521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.602005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.603527 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:58.604053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.606841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.634841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 272/555] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:58.634841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.660743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.731284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91778e-b1b8-4a25-b0f5-d3f87cd02b26 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.749625 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81666-7a17-4536-a7c8-898fe3c4c926 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.770446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.771249 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.771803 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.772412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.773155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:58.773563 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.775235 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.798146 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.799521 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 284/556] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:58.799861 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.800305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.800697 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.800938 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.801986 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:58.802500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.804078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.816982 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.829043 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 273/557] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:58.829043 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:58.925552 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900728-5b8a-4bd3-8527-85e1913efba9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.933987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:58.951167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8070-ecad-47f1-95f7-86bfc237ed7c None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:58.964359 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.965274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.965799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.967384 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.979317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:58.981271 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:58.992272 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:58.992920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:58.993209 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:58.993492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:58.998733 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:58.999826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:59.000821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:59.000821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:59.000821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:59.007535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.007535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.008310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 285/558] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:59.009548 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:40:59.025273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:59.026417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:40:59.028100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:40:59.028100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:40:59.028100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:59.031252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 274/559] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:59.032071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.102563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.127961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e18d-d756-4c25-95e8-6a2f474b9531 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.166822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba682e32-c477-41d5-b344-50be01b5fd97 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.169018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.169894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.170387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.170905 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] PATH_INFO: `/v3/users/nova` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.180780 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.182671 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:40:59.196517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:59.197688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:40:59.198804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:59.206927 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 286/560] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:59.208369 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.211333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.212105 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.212403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.212684 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.213380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:59.214507 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.215200 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.229743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 275/561] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:59.230688 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.231765 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.319577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58057bad-e189-4dd5-a949-1dc467c3e270 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.349106 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0aefc3-403b-4e63-aaf1-530525c7daf4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.361822 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.362764 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.362911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.363315 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.364412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.365439 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:59.365849 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.367810 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.381313 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 287/562] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:59.382216 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.385412 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.387198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.387518 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.387801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.388536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:59.388918 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.390663 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be8b23-9c0a-4fc3-aaac-29bba6982f2f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.414673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 276/563] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:59.414673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7d798-bd67-4ef9-903c-fa8c67c44e91 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.414673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7d798-bd67-4ef9-903c-fa8c67c44e91 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.414673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7d798-bd67-4ef9-903c-fa8c67c44e91 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.414673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 277/564] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:40:59.415184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7d798-bd67-4ef9-903c-fa8c67c44e91 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.470781 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.492884 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ebbdf-ae3c-409f-bea7-28c98f7748dc None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.525919 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7d798-bd67-4ef9-903c-fa8c67c44e91 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.530066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.530770 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.531125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.531501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.538293 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.539837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:59.552461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:59.553662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 288/565] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:59.554704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.570129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.570987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.571331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.571619 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.578765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.580291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-0393cc3e-c2fd-4845-8ade-03fceba3056d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:40:59.593971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:40:59.594973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 278/566] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.596628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.597303 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.615910 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:40:59.617118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:40:59.642970 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.666725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d32b3-86ae-424d-ba59-4645fe43875f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.699333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 279/567] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:40:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:40:59.699333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.709405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.710099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.710508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.710797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.719842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.786175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.788745 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.827787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c70f32-6ece-45b2-82ce-980f2481934a None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:40:59.872636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:40:59.873677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:40:59.874240 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:40:59.874773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:40:59.875782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:40:59.876307 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:40:59.878214 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:40:59.893853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 280/568] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:40:59.895010 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.919628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:40:59.919931 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 289/569] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:40:59.921175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:40:59.969394 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:00.007891 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99460fc9-615d-41f1-bff8-6fb3db6271b6 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.029936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7e408-d13f-44e0-a87c-f67b7fa1c947 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.044277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.044820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.045106 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.045384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.046075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:00.046386 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.048097 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.065491 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 281/570] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:00.065491 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.072397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.073129 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.073555 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.073852 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/136ade6f493b4b069b7d2710b4ab50ae/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.090992 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=136ade6f493b4b069b7d2710b4ab50ae, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.095929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.145379 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:00.165008 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300289-6bc5-44c7-a8d3-4d53ad616945 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.180331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 290/571] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:40:59 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/136ade6f493b4b069b7d2710b4ab50ae/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:41:00.181443 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.222528 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.224189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.225198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.225796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.240696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.243350 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:00.265711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:00.267543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:00.268571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:00.268571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:00.268571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:00.269120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 282/572] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:00.270211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.306301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:00.321233 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf45fc-1d4e-4607-8e42-a6d392eeaaeb None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.365912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.366985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.367529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.368017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.368814 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.370375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:00.380419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:00.381547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:00.382496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 291/573] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:00.384097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.402338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210240-565b-4a1f-8134-d6419a7a3934 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.436807 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:00.443287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb865b4-dca6-443d-9f21-91881f9181c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.444120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb865b4-dca6-443d-9f21-91881f9181c0 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.444382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb865b4-dca6-443d-9f21-91881f9181c0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.444723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb865b4-dca6-443d-9f21-91881f9181c0 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/6639d3279dfc42dba4073a200a09a357/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.447415 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb865b4-dca6-443d-9f21-91881f9181c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=6639d3279dfc42dba4073a200a09a357, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.471829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb865b4-dca6-443d-9f21-91881f9181c0 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.520527 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b31145-d624-44a1-a4ea-4055e5822556 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.554616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.555227 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.555545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.555866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.556574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:00.556830 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.558194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.570659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 292/574] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:00.571499 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.586767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:00.667176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 283/575] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:41:00 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/6639d3279dfc42dba4073a200a09a357/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:41:00.667176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc1be-c3e4-4a68-81e0-5ed4e21d28f5 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.667176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc1be-c3e4-4a68-81e0-5ed4e21d28f5 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.667176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc1be-c3e4-4a68-81e0-5ed4e21d28f5 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.667176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 284/576] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:00.667176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fc1be-c3e4-4a68-81e0-5ed4e21d28f5 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.679968 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ac8a6-aa5c-4274-9479-422aad406119 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.720484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.721238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.721523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.721801 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.722465 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:00.722816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.724304 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d8aa9-0b37-46ea-bb14-c6dac7cbe5b9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.729878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 293/577] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:00.731206 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.731530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.731808 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.753238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:00.754472 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:00.807059 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fc1be-c3e4-4a68-81e0-5ed4e21d28f5 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:00.843677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 294/578] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:00.845003 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.853198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:00.853198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:00.853198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:00.853198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:00.860227 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:00.867734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:00.867734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:00.880873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:00.882333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:00.883460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:00.883967 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 285/579] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:00.889625 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:00.963757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:01.017894 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a76724-ea29-47f0-9595-31f05b70c99e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.025418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bb243-d379-4cf4-8a6a-374d8b9f525d None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.056361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.057171 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.057409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.057691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.058394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:01.058713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.063331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.074407 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 286/580] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:01.081133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:01.156050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.156694 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.156985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.157264 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.158017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:01.158315 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.159914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.166570 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 295/581] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:41:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:01.167410 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:01.201472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f730e77-0fd3-451e-ac57-d8d699009723 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.241757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.243190 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.243825 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.244329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.245344 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=921c6749f51c49d78748876ec65bd71f) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:01.245901 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.246015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:01.251523 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:01.261747 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.261747 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 287/582] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/role_assignments?user.id=921c6749f51c49d78748876ec65bd71f&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:01.261747 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:01.302320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a67f-4494-40d2-b02c-f97679fe3d48 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.376805 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.378496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.378778 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.379053 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] PATH_INFO: `/v3/services/network` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.379688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.381105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:01.391188 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:01.392524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:01.393512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 296/583] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:01.405477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:01.459541 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abd281-6c96-41a2-bd41-3df6265b90d3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.505761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.506788 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.507096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.507411 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.508113 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] RBAC: Adding query filter params (interface=public, service_id=c240533e6bff46fcac7cd02a05ff85e8, region_id=RegionOne) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:01.508449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.510159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.524922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 288/584] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/endpoints?service_id=c240533e6bff46fcac7cd02a05ff85e8&interface=public®ion_id=RegionOne => generated 194 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:01.531598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:01.586977 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:01.599286 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f8b8-0228-43a4-b1ea-3e5a05fd746f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.647137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.647724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.648011 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.648290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.649187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] RBAC: Adding query filter params (name=network) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:01.649501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.651335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.652754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:01.663205 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 297/585] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:01.671631 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:01.671998 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627fd57-888a-4759-ad3e-fc1cfe740fc9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.749668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.750749 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.751330 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.751845 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.752686 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.754475 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.769457 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 289/586] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:01.802002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d4d62-b10a-4f03-8716-ca1ba0215e7c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:01.861877 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:01.862526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:01.862811 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:01.863089 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:01.863804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:01.864117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:01.865554 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:01.876076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 298/587] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:01.881833 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:02.052143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bc6d7-1f0b-4a71-b134-cc410d48cd88 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:02.055270 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:02.089471 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:02.090103 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:02.090390 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:02.090671 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:02.091414 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] RBAC: Adding query filter params (interface=public, service_id=63020d52016c4c31b78a67b9804809fc, region_id=RegionOne) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:02.091733 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:02.093112 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bf760-cf41-4218-a742-41cdfd2ab387 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:02.099571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 290/588] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:01 2022] GET /identity/v3/endpoints?service_id=63020d52016c4c31b78a67b9804809fc&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:02.105410 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:02.189408 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:02.243415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf49af-b2f1-4de0-aec0-1ec4f6a8c9d2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:02.296029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cd9dd-5719-4129-ade3-5e811fa3420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:02.296959 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd9dd-5719-4129-ade3-5e811fa3420c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:02.297236 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd9dd-5719-4129-ade3-5e811fa3420c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:02.297461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd9dd-5719-4129-ade3-5e811fa3420c None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:02.298084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cd9dd-5719-4129-ade3-5e811fa3420c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:02.300043 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cd9dd-5719-4129-ade3-5e811fa3420c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:02.305912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 299/589] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:02 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:02.598100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e135d-0938-4624-ad8f-4d666ec4e788 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:02.598649 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e135d-0938-4624-ad8f-4d666ec4e788 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:02.598649 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e135d-0938-4624-ad8f-4d666ec4e788 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:02.600305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 291/590] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:02.606251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:02.606591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:02.607135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:02.631872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:02.633097 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:02.720552 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 300/591] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:03.063049 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e135d-0938-4624-ad8f-4d666ec4e788 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:03.197089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e135d-0938-4624-ad8f-4d666ec4e788 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:03.242815 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69e335-5d4b-4401-85ae-da7883425beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:03.243494 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69e335-5d4b-4401-85ae-da7883425beb None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.243782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69e335-5d4b-4401-85ae-da7883425beb None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.244060 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69e335-5d4b-4401-85ae-da7883425beb None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.258839 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69e335-5d4b-4401-85ae-da7883425beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-4d69e335-5d4b-4401-85ae-da7883425beb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:03.263740 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:03.264793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:03.265766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 292/592] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:03.270792 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:03.354977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:03.414857 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739043c3-daf7-4ce4-a1f5-5d79c6c0d50d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.415099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d297f1f-518c-4275-bcb3-9017d94af64e None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:03.415201 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739043c3-daf7-4ce4-a1f5-5d79c6c0d50d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.415500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739043c3-daf7-4ce4-a1f5-5d79c6c0d50d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.417730 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 293/593] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:03.423822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.424366 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.424860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.459107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:03.460349 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:03.479471 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:03.480436 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.480987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.481436 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.482278 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:03.482738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:03.484407 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:03.491702 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:03.502565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 301/594] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:41:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:03.512731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:03.536646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 294/595] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:03.629905 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68641e83-34be-4233-b757-56573ba1da93 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:03.654520 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:03.677417 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:03.678101 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.678392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.678672 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:03.696498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:03.711186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:03.712514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:03.713748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:03.713748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:03.713748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:03.713748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 302/596] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:03.718139 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:03.879375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eef21-97af-4957-9f62-4dbdd4118b87 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:03.914749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:03.936053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:03.936672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.936957 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.937237 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.937889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:03.938196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:03.940157 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0484fb-d853-4763-9ea6-91fd88836f66 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:03.954526 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 295/597] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:03.956279 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ede4fd-9b35-45b5-a657-75d39ebd15b0 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.956565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ede4fd-9b35-45b5-a657-75d39ebd15b0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.956843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ede4fd-9b35-45b5-a657-75d39ebd15b0 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.958189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 296/598] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:03.964229 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:03.964517 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:03.964791 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:03.983113 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:03.984334 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:03.998547 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.056001 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55333ae-8c0a-47df-94d3-b262f12fbc44 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.056232 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 297/599] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:04.057176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.102597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.103742 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.104558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.105038 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.121767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:04.129566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:04.130630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:04.131630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:04.134699 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 303/600] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:04.141679 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.175293 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b122f9f-d7b0-45ac-ad16-41977e6417ae None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.200263 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.222559 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.224576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.226532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.227025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.241697 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.247816 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:04.269014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:04.270156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:41:04.271269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:04.281776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 298/601] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:04.287871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.309843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.340600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7c93e-b95b-4505-992e-e74f8decea12 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.404367 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.405075 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.405244 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.405522 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.406190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:04.406497 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.408477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.417371 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 304/602] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:04.418179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.439208 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855a996-5e45-4134-ad57-2ad15b4aeea2 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.480579 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.481734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.482608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.483348 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.484298 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:04.484793 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.486651 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.511418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 299/603] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:04.512627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.513972 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.523622 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e2668-88ba-4b0d-a717-076fb057b00b None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.559021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.559793 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.560143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.560486 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.561145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.562587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:04.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:04.573983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:04.575267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:04.580187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 305/604] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:04.581053 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.597842 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.641147 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852573c4-938b-4cb7-92fe-9db523d33ce0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.681865 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.683061 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.683550 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.684018 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.697126 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.698616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5bbd0-4ade-4ae1-a2d0-9cdf143a6b74 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.699072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:04.718160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:04.720011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:04.721247 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:04.721247 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:04.721247 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:04.723655 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 300/605] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:04.724693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.736702 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.737314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.737598 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.737889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.765825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.817425 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.818891 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.838581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fd19d-8ac0-4c41-a7ae-38ede25e3aa9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:04.857812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:04.876107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:04.878340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.878340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.878340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.878646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:04.878879 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:04.880298 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01097f4-6ebe-4226-bb1e-86e69aef0766 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:04.889542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 301/606] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:04.889542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da61f84-9116-41b8-abe6-5d26bdc751b9 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:04.889542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da61f84-9116-41b8-abe6-5d26bdc751b9 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:04.889542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da61f84-9116-41b8-abe6-5d26bdc751b9 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:04.889542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 302/607] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:04.889542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da61f84-9116-41b8-abe6-5d26bdc751b9 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.930358 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 306/608] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:41:04.931667 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:04.992040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.008289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da61f84-9116-41b8-abe6-5d26bdc751b9 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.040505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f93e8b-0a12-4ee9-b109-d6230b2461b4 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.047957 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.049151 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.049442 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.049723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.050378 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:05.050690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.052109 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529125d-4768-482c-96b1-8b97b276f3ad None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.064931 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 303/609] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:05.066990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.067548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.068048 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.078255 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.078867 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.079157 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.079480 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.080119 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:05.080426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.081798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.095618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:05.095936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 307/610] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:05.095936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.096136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:05.139789 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.178997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 304/611] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:05.180160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.233287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d0d49-4959-492c-b47d-5411f150eb87 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.265524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.284890 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.285533 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.285815 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.286093 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.298958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.300469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.306342 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6bd04-a06d-416c-91f8-716061ffd4f1 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-7a981329-f1e8-48ee-8c64-ca4a75807d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:05.317813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:05.318912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:05.320008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:05.320008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:05.320008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:05.323587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 308/612] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:05.324847 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf1dd0-c160-44c6-b3b8-f6659c62aa79 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.325137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf1dd0-c160-44c6-b3b8-f6659c62aa79 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.325419 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf1dd0-c160-44c6-b3b8-f6659c62aa79 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.326842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 309/613] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:05.327416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf1dd0-c160-44c6-b3b8-f6659c62aa79 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.356031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.356678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.356966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.357239 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.366635 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.368387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ac8cd5a8-0a20-473f-9a1d-86f89c4dcd10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:05.386859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:05.388026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:05.389115 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 305/614] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:05.389814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.389814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.390501 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.414410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:05.415897 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:05.450741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf1dd0-c160-44c6-b3b8-f6659c62aa79 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.452324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.487655 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.488540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.488829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.489113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.489771 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:05.490086 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.491486 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.503817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 306/615] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:05.503817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.511250 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 310/616] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:05.511250 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.561986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.635070 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadbe5a-b6b3-4b8b-8870-62991082f60f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.646007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b14e8-8c42-48bb-a21a-69b3ce7fa405 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.676631 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.677655 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.678374 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.678834 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.680165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.682093 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.690418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.691104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.691512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.691857 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.692668 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:05.693047 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.694531 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:05.695033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:05.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 05:41:05.697523 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:05.701137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 311/617] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:05.702143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.724996 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 307/618] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:05.727026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.758356 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.816935 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:05.823192 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7770c-7500-460f-a56c-6404aeb84339 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.850480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad213e5-e85c-46b9-a9db-33b54844e42c None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.857644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.858287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.858578 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.858862 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.866432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.867917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:05.879247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:05.880536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 312/619] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:05.881725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.893641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:05.894262 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:05.894554 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:05.894833 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:05.895516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:05.895833 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:05.897193 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:05.906520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 308/620] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:05.906520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:05.994162 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c96a1-59ed-48ab-abe8-e732f3426e17 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:05.999033 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.017237 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b79b04-29ff-4071-835c-63cc6d2ca963 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.030551 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.031180 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.031505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.031791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.057469 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.058263 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.058675 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.058962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] PATH_INFO: `/v3/services/network` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.059590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.060961 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.061106 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:06.073609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:06.074696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 05:41:06.075838 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:06.078235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 309/621] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:06.079175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.112263 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.113726 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.191620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73a1bb-feb8-4c2e-ae73-aa06755c722c None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.222847 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 313/622] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:41:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:41:06.224067 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.232854 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.233482 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.234128 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.234417 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.235462 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:06.235841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.237464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.250744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 310/623] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:06.251651 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.261757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.287682 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.332541 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1364-b922-4c21-bdb1-117b6c1296be None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.361591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6465ec2-0e46-42f7-ba58-3a386c03ddba None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.367730 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.368350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.368634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.369332 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.369976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:06.370290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.372049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.377917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 314/624] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:06.378629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.402009 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.402627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.402914 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.403191 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.403937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] RBAC: Adding query filter params (name=network) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:06.404294 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.405614 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.410567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 311/625] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:06.412361 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.489208 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebd2a6-dde8-4a5c-a1e6-1c004d1f1cb8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.489760 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.514175 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.520213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade0f6-4a47-4d2d-ae52-759f6ba8669b None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.529409 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.530195 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.530483 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.530763 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.541992 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.543477 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.556183 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.556789 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.557075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.557351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.558072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=136ade6f493b4b069b7d2710b4ab50ae) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:06.558385 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:06.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:06.560039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:06.561281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:06.561281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:06.561281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:06.561701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.575744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 312/626] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/role_assignments?user.id=136ade6f493b4b069b7d2710b4ab50ae&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:06.575744 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.576073 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 315/627] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:06.576073 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.695540 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7fa04-9794-46b6-a124-621e48aaef7e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.709597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99f0cb-b627-4179-ba57-bb1a422a8ed3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.737659 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.738285 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.738574 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.738854 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.739570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.741092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.748284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.759282 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.760222 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.760785 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.760895 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.761510 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] RBAC: Adding query filter params (type=network) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:06.761824 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.763232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:06.769402 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 316/628] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:06.770172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.815277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:06.884062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 313/629] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:06.885568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:06.895332 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3c4fa-91b7-45db-b132-8a4d9bd8bebe None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:06.984663 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:06.985321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:06.986217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:06.989479 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:06.990422 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:06.990823 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:06.992335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:07.022344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 317/630] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:07.028496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:07.113817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476509-ca2c-4dc1-a0db-040f1485143f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:07.154954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba3e0-6fe4-4d7a-a9eb-93c0bebcff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:07.155718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba3e0-6fe4-4d7a-a9eb-93c0bebcff3b None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:07.156134 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba3e0-6fe4-4d7a-a9eb-93c0bebcff3b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:07.156575 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba3e0-6fe4-4d7a-a9eb-93c0bebcff3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:07.157889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaba3e0-6fe4-4d7a-a9eb-93c0bebcff3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:07.160270 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaba3e0-6fe4-4d7a-a9eb-93c0bebcff3b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:07.182375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:07.195977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b839a7-6864-4d07-beb6-231a180f1637 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:07.234918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:07.236301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:07.236588 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:07.236865 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:07.244800 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:07.246428 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:07.259146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:07.260380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:07.261731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 318/631] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:07.267059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:07.290886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 314/632] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:06 2022] POST /identity/v3/endpoints => generated 337 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:07.345808 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a735fe-bde5-4d41-981c-8672373417ff None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:07.346126 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a735fe-bde5-4d41-981c-8672373417ff None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:07.346377 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a735fe-bde5-4d41-981c-8672373417ff None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:07.348079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 315/633] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:07.353695 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:07.353984 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:07.354267 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:07.374973 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:07.376880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:07.418610 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f342bd3-77eb-4afe-9c54-72c3d563b630 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:07.445403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:07.467769 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:07.468765 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:07.469266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:07.469742 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:07.470696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:07.471221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:07.472940 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:07.487797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 319/634] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:07.494358 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:07.500018 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 316/635] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:07.515811 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:07.608619 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5607a47-9c12-43f6-87ff-09b57a333837 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:07.647458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:07.648453 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:07.648932 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:07.649413 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:07.650368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=6639d3279dfc42dba4073a200a09a357) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:07.650895 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:07.654315 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be808c-c8c6-418b-9a12-5afa64aabb58 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:07.661641 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 320/636] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:07 2022] GET /identity/v3/role_assignments?user.id=6639d3279dfc42dba4073a200a09a357&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:07.867524 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:07.973990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39270e-6684-4607-b727-f3f18173bbe8 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:08.013578 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1cda6-7fd6-41e0-99de-3e66ff283fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:08.014521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1cda6-7fd6-41e0-99de-3e66ff283fa2 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:08.014997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1cda6-7fd6-41e0-99de-3e66ff283fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:08.015505 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1cda6-7fd6-41e0-99de-3e66ff283fa2 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:08.016400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1cda6-7fd6-41e0-99de-3e66ff283fa2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:08.018585 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1cda6-7fd6-41e0-99de-3e66ff283fa2 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:08.107107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 317/637] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:07 2022] POST /identity/v3/services => generated 269 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:08.968622 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb809e-1e0d-4d89-9cf4-dbc09a02c3a0 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:08.969652 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb809e-1e0d-4d89-9cf4-dbc09a02c3a0 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:08.970702 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb809e-1e0d-4d89-9cf4-dbc09a02c3a0 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:08.972598 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 321/638] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:08.978202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9606a1-8e9e-4540-800f-e9b724ec617d None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:08.979045 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9606a1-8e9e-4540-800f-e9b724ec617d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:08.979571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9606a1-8e9e-4540-800f-e9b724ec617d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.000347 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-8a9606a1-8e9e-4540-800f-e9b724ec617d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:09.002039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-8a9606a1-8e9e-4540-800f-e9b724ec617d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:09.035475 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:09.091408 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 318/639] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:08 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:09.280454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d252-75f4-4550-bf94-8311df3fdcc5 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:09.280838 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d252-75f4-4550-bf94-8311df3fdcc5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:09.281031 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585d252-75f4-4550-bf94-8311df3fdcc5 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.283138 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 322/640] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:09.288447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:09.288743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:09.289022 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.307506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:09.308693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:09.373870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585d252-75f4-4550-bf94-8311df3fdcc5 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:09.378348 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 319/641] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:09 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:09.454818 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:09.484886 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585d252-75f4-4550-bf94-8311df3fdcc5 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:09.522270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:09.522917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:09.523201 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:09.523510 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.534327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:09.539149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:09.540202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:09.541308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 323/642] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:41:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:09.548974 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:09.645253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:09.650029 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:09.656765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f73e7-b2da-47e9-9666-7b5ec5b14701 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:09.695430 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:09.696055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:09.696343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:09.696626 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.697310 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:09.697623 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:09.699003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:09.711030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 320/643] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:41:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:09.717133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:09.784243 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fdbc4-bf5f-4970-b15e-e338dc0f6595 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:09.830281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:09.831645 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fca6c-9af7-4eda-9ec3-98a2f01e7d9f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:09.836582 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:09.837498 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:09.837954 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:09.838404 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.839327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:09.841505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:09.851825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:09.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:09.854455 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:09.857246 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 324/644] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:09.866446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:09.871622 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:09.872285 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:09.872572 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:09.872850 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:09.886144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:09.887647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:09.903534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:09.904691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:09.906039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:09.906039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:09.906039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:09.906039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 321/645] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:09.916259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:09.933183 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:09.978320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc12ab-6be0-4505-aa69-701313d050a0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.016491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.017136 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.017428 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.017711 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.018348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:10.018663 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.020072 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029a48b-7b33-44d7-87e1-0286528e6a5f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.029105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 325/646] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:10.029105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c29c-2a7f-400a-a52b-26a394d631ad None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.029105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c29c-2a7f-400a-a52b-26a394d631ad None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.029105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c29c-2a7f-400a-a52b-26a394d631ad None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.029105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 326/647] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:10.032294 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c29c-2a7f-400a-a52b-26a394d631ad None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.037773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fac0d-ee34-475d-93a5-0e794cc9a739 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.073272 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.073882 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.074168 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.074452 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.075095 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:10.075437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.076806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b59e3-17d0-44fa-a4f2-0aa8e86c906f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.093972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 322/648] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:10.093972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.093972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.093972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.103926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:10.111184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:10.112418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:10.148232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c29c-2a7f-400a-a52b-26a394d631ad None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.180386 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:10.183704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.184325 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.184615 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.184898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.185567 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:10.191598 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.191768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 323/649] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:10 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:10.191768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.191955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a418290-41da-45f9-b220-4317e72c3803 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.195342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 327/650] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:10.330860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540bb7e-3fe6-422f-a8c8-50596a349a0a None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.370774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.371144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f3c4-5012-4826-9787-b162ecbf5eab None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.371422 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f3c4-5012-4826-9787-b162ecbf5eab None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.371528 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.371705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f3c4-5012-4826-9787-b162ecbf5eab None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.371806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.371999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.373186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 328/651] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:10.378701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.378991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.379266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.380109 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.381590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:10.394963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:10.396101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:10.397175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 324/652] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:10.402253 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.404893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:10.406099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:10.472586 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:10.483266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 329/653] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:10 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:10.486050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.519408 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe83fc7-5db3-4384-921a-a3c031d674bd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.557489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:10.559382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.560212 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.560687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.561169 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.562101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:10.562614 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.564401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.587249 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 325/654] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:10.593291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.614063 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f9374-fabe-4696-b2bb-1c1d8cd452a5 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.651137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.651795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.652085 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.652370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.652958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.654345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:10.664450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:10.665627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:10.667019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 330/655] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:10.679353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.726774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89105c-b87f-481c-8e6a-6e6cd629164d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.749839 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:10.771777 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.772482 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.772773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.773321 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.783330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7ce1-d218-47cd-a682-b9b910e8e8b2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.787278 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.788778 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.807887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:10.808054 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:10.809081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:10.810094 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:10.810094 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:10.810094 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:10.810094 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 326/656] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:10.810389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.827857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.828500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.828806 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.829687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.830335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:10.830650 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.832062 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.841399 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 331/657] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:10.841399 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:10.930246 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5edb1f-8af5-4319-a03a-9ff7da0f9c78 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.943906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59144-1325-49d9-9a1f-b7808f75aa74 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:10.968419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.969067 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.969355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.969638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.970267 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.971750 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.979719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:10.980284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:10.980560 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:10.980848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:10.981374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:10.982399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:10.983909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:10.984759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:10.984759 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:10.984921 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 327/658] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:10.985006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:10.985092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.001637 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 332/659] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:11.001637 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.020564 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:11.049501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:11.094989 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64a619-2fad-4f5c-8c5d-913e7a5042d8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.110688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac70ade-82bc-433c-86f2-c8f1f5289565 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.131954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.132617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.132912 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.133197 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.133962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:11.134693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.135705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.143464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 328/660] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:11.143464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.163987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.165017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.165830 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.166814 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.167787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:11.168347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.170792 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.179696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 333/661] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:11.186460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.271034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:11.277845 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d0025-8cc8-46e8-8eff-96b405a7e985 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.317762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.318413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.318699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.319261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.326826 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.328332 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.331749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11fdc-28f9-4f96-9175-a799884a3d5a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.339965 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:11.340538 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:11.341530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:11.343210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 329/662] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:11.350092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.379629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.380545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.380689 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.380920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.381575 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:11.381888 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.383322 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.389059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 334/663] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:11.394450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.479702 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ef541-9f04-4b80-aa8d-1708abde588e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.522296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.523271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.523772 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.524337 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.525398 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:11.526216 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.528248 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.528837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402398e-6204-49c5-84cf-20c4a5645156 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.548411 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 330/664] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:11.550014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.571603 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:11.578525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.579250 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.587080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.587080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.587080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] RBAC: Adding query filter params (interface=public, service_id=875876d845cb4dea9c839dfb9d47c8eb, region_id=RegionOne) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:11.587080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.587080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.597564 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 335/665] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:11 2022] GET /identity/v3/endpoints?service_id=875876d845cb4dea9c839dfb9d47c8eb&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:11.602784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:11.653249 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:11.665563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896df513-a473-42b0-b86a-22763aa3a6e1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.711352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abbdde-31f6-4d34-8c68-f041a37e4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.712637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abbdde-31f6-4d34-8c68-f041a37e4559 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.713363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abbdde-31f6-4d34-8c68-f041a37e4559 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.714192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abbdde-31f6-4d34-8c68-f041a37e4559 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/136ade6f493b4b069b7d2710b4ab50ae/roles/d5f3d562e3ac4f54b99a938c6e104cd4` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.716691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abbdde-31f6-4d34-8c68-f041a37e4559 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=136ade6f493b4b069b7d2710b4ab50ae, role_id=d5f3d562e3ac4f54b99a938c6e104cd4)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.719197 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abbdde-31f6-4d34-8c68-f041a37e4559 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.731221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d91ef-9a56-49ee-a4c4-152a13f3d7ee None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:11.767501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18a4b4-2ab3-4617-8671-82992eb525c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:11.768117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a4b4-2ab3-4617-8671-82992eb525c6 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:11.768406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a4b4-2ab3-4617-8671-82992eb525c6 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:11.768690 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a4b4-2ab3-4617-8671-82992eb525c6 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:11.769306 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18a4b4-2ab3-4617-8671-82992eb525c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:11.770795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18a4b4-2ab3-4617-8671-82992eb525c6 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:11.776710 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 336/666] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:11 2022] GET /identity/v3/services => generated 1029 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 05:41:11.804156 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 331/667] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:41:11 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/136ade6f493b4b069b7d2710b4ab50ae/roles/d5f3d562e3ac4f54b99a938c6e104cd4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:41:12.458251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8b32c-e1fa-431d-89fa-61616f1fc1c2 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:12.458828 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8b32c-e1fa-431d-89fa-61616f1fc1c2 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:12.458828 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8b32c-e1fa-431d-89fa-61616f1fc1c2 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:12.460471 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 337/668] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:12.466122 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1a7e5-fd84-4f36-8620-52d73918aa79 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:12.466593 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1a7e5-fd84-4f36-8620-52d73918aa79 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:12.466704 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1a7e5-fd84-4f36-8620-52d73918aa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:12.485136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-08f1a7e5-fd84-4f36-8620-52d73918aa79 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:12.486382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-08f1a7e5-fd84-4f36-8620-52d73918aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:12.557582 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 332/669] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:12 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:12.826870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8b32c-e1fa-431d-89fa-61616f1fc1c2 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:12.933644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:12.939258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8b32c-e1fa-431d-89fa-61616f1fc1c2 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:12.975244 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d893ae-6163-4e79-b899-6874b08c7f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:12.976143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d893ae-6163-4e79-b899-6874b08c7f39 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:12.976614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d893ae-6163-4e79-b899-6874b08c7f39 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:12.977040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d893ae-6163-4e79-b899-6874b08c7f39 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:12.977924 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d893ae-6163-4e79-b899-6874b08c7f39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:12.979500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d893ae-6163-4e79-b899-6874b08c7f39 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:13.061520 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 338/670] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:12 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:13.353929 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2729ada-e1df-45ca-bd30-e8abed0d9677 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:13.354346 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2729ada-e1df-45ca-bd30-e8abed0d9677 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:13.354544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2729ada-e1df-45ca-bd30-e8abed0d9677 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:13.356291 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 333/671] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:13.362010 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf962fb-5293-4d5a-a28d-725ca9673a3d None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:13.362309 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf962fb-5293-4d5a-a28d-725ca9673a3d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:13.362597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf962fb-5293-4d5a-a28d-725ca9673a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:13.381833 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-1bf962fb-5293-4d5a-a28d-725ca9673a3d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:13.383059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-1bf962fb-5293-4d5a-a28d-725ca9673a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:13.462153 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 339/672] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:13.732531 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2729ada-e1df-45ca-bd30-e8abed0d9677 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:13.742160 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:13.850217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2729ada-e1df-45ca-bd30-e8abed0d9677 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:13.887232 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaa570-51a3-4dd3-89dd-5144afbfece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:13.887880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaa570-51a3-4dd3-89dd-5144afbfece9 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:13.888167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaa570-51a3-4dd3-89dd-5144afbfece9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:13.888446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaa570-51a3-4dd3-89dd-5144afbfece9 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:13.889124 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaa570-51a3-4dd3-89dd-5144afbfece9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:13.890516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaa570-51a3-4dd3-89dd-5144afbfece9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:13.966135 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 334/673] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:13 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:14.089612 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59fca1-f57b-4846-827d-5ecd768fdc07 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.090079 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59fca1-f57b-4846-827d-5ecd768fdc07 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.090463 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59fca1-f57b-4846-827d-5ecd768fdc07 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.092376 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 340/674] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:14.098725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a2688-b14c-4ed7-9098-87c875e9ea8f None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.098927 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.099179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.099432 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.099946 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a2688-b14c-4ed7-9098-87c875e9ea8f None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.100449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a2688-b14c-4ed7-9098-87c875e9ea8f None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.102463 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 335/675] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:14.109000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.109000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.109000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.121466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:14.123108 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:14.129849 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:14.131340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:14.212065 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:14.214712 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 341/676] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:14.222900 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 336/677] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:14.488991 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:14.494602 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:14.501944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:14.605970 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7876-c3af-4e50-a577-bd7649048114 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:14.617935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73ee4a-c00c-4ee1-b558-cfac33cb7398 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:14.644960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:14.645855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.646493 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.647009 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.658638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:14.659020 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:14.659801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.660246 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.660681 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.661435 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:14.662963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:14.664955 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:14.666022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:14.667023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:14.669953 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 342/678] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:41:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:14.677884 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:14.678446 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:14.679759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 05:41:14.680858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:14.684985 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 337/679] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 05:41:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:14.690589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:14.761548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:14.766342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:14.801987 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e7b98-86aa-4158-9bf9-2473e38c3b52 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:14.825876 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361639b8-48b9-44e5-81ca-32d698ab3e29 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:14.842480 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:14.843399 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.844123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.845114 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.846445 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:14.847199 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:14.849061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:14.866223 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 343/680] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:41:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:14.868075 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:14.873046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:14.873970 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:14.874464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:14.874979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:14.875910 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:14.876448 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:14.878055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:14.888421 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 338/681] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:14.889722 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:14.984512 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559006-70c2-471c-811f-bd0e27558db5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.016365 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:15.022239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.023291 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.026711 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.026711 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.028153 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abc57a-d8e4-41cb-8add-1d85a98f5e51 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.039223 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.042260 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.054755 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:15.064592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:15.067799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:15.068932 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:15.068932 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:15.068932 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:15.068932 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 344/682] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:15.071674 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.071948 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.072886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.073387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.073891 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.075068 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:15.076671 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.078313 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.085201 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 339/683] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:15.091674 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.186177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab96c-62ab-4c6f-a894-2828224b610d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.222405 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.223127 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.223506 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.223842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.224590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:15.224970 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.226428 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.235806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85cee3-d2af-411d-833d-d2d5a10fce13 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.243675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 345/684] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:15.245330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.279473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a19f15-e5a7-4c3b-b199-b9a8010fb793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.280072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a19f15-e5a7-4c3b-b199-b9a8010fb793 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.280326 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a19f15-e5a7-4c3b-b199-b9a8010fb793 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.280610 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a19f15-e5a7-4c3b-b199-b9a8010fb793 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.281292 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a19f15-e5a7-4c3b-b199-b9a8010fb793 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.282713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a19f15-e5a7-4c3b-b199-b9a8010fb793 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.294774 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:15.347443 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2feb5-7102-4298-a43c-8ebbc8925653 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.381467 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.382105 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.382397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.382681 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.390176 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.398062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 340/685] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:15.398260 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.398372 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e5c47-add8-4142-a292-b2ea07ea88fa None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.398372 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e5c47-add8-4142-a292-b2ea07ea88fa None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.398372 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e5c47-add8-4142-a292-b2ea07ea88fa None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.398372 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 341/686] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:15.400287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.400579 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.400858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:15.412790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:15.414161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 05:41:15.415284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:15.418985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 346/687] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:15.424383 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.424597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:15.425865 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:15.475663 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:15.519462 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 342/688] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:15.572906 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389cd7-cc4f-40b2-b47d-64d247182bdb None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.575635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:15.612481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.613642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.614149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.614666 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.615501 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:15.615828 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.617347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.636443 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 347/689] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:15.642486 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.750425 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55483844-229e-4b28-9c53-f0cc5441e4f4 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.788395 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.789481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.789765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.790072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.806957 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.834070 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:15.865996 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.867543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.922474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9b8b2-1379-481c-bf63-ee4a44b0d662 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:15.964745 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:15.966285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:15.966796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:15.967322 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:15.968155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:15.970698 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:15.982796 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 343/690] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:15.983226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:15.984469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:15.986015 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:15.988554 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:15.993911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 348/691] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:15.994727 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.041037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.071869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.108295 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32849-d573-495d-8af9-48a70d138ca9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.147252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3a0b5-b5bf-4bf9-91b0-fb61a73a3603 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.158345 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.159746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.160051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.160339 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.174560 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.176368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:16.194473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:16.195736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:16.196762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:16.196762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:16.196762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:16.196762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 344/692] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:16.199951 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.200883 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.201179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.201737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.202237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.203100 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:16.203643 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.206412 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.217191 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 349/693] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:16.220556 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.318988 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7a8e7-ba93-409d-b32c-e1abdaf9c309 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.329735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.338647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f77649-56ed-4836-ab85-6dc4a6c92adc None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.343751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.355243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.355951 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.356238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.356518 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.357161 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:16.357470 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.358902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15917406-a4f7-4076-972d-61990d25f043 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.372586 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 345/694] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:16.373978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24db1-b4bd-475b-a9c8-92f18a525da8 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.374278 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24db1-b4bd-475b-a9c8-92f18a525da8 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.376430 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24db1-b4bd-475b-a9c8-92f18a525da8 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.376430 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 346/695] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:16.378781 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd24db1-b4bd-475b-a9c8-92f18a525da8 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.380046 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.380697 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.380982 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.381259 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.381920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:16.382380 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.383767 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff374a-e0ae-4ffe-b3f8-f23653de2a60 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.389686 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 350/696] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:16.391440 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.392025 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.392553 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.412132 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:16.413329 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:16.485893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 351/697] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:16.486980 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.504535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd24db1-b4bd-475b-a9c8-92f18a525da8 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.551118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.552265 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.553055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.553788 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.561766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.563996 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.576263 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:16.579040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:16.580078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:16.581092 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:16.588863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 347/698] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:16.590567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.595256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925680ff-4cef-48ca-8fd6-6be503c2674b None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.635336 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.635933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.636225 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.636561 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.637347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] RBAC: Adding query filter params (interface=public, service_id=9a90de35b9e34aa3a9d22fab7dd0bc4d, region_id=RegionOne) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:16.637671 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.639258 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.643771 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.649513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 352/699] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/endpoints?service_id=9a90de35b9e34aa3a9d22fab7dd0bc4d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:16.650418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.707877 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d7e9c-3aef-4c64-af25-0c240dd7ecda None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.748755 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.749765 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.750258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.750831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.752296 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:16.753105 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.754959 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.764190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fabf4-0300-4a9a-8c4d-e81d025a687c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.769802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 348/700] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:16.770947 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.811856 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.812480 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.812772 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.813056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.813721 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.815090 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.821344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 353/701] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 05:41:16.824461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.856207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.890046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358ff2d-1a10-4d8c-9dc5-9795799da753 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:16.892300 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:16.929040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:16.930091 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:16.930587 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:16.931072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:16.932995 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:16.934718 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:16.945393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:16.946613 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:16.947605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:16.950599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 349/702] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:16.957295 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:16.985846 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be888e7-0e00-447a-99f3-4af1881a28a9 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:17.032745 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:17.033369 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.033659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.033944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.034887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] RBAC: Adding query filter params (role.id=d5f3d562e3ac4f54b99a938c6e104cd4, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=136ade6f493b4b069b7d2710b4ab50ae) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:17.035211 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:17.037495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:17.045396 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 354/703] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/role_assignments?user.id=136ade6f493b4b069b7d2710b4ab50ae&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=d5f3d562e3ac4f54b99a938c6e104cd4 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:17.077199 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c98ad-13f4-458e-8039-74b8ccffa13e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:17.122043 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:17.122689 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.122981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.123267 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.123968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:17.124286 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:17.125874 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:17.132184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 350/704] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:17.138437 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:17.233876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:17.259460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0b8c8-784f-4ad3-943b-38977a727ab1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:17.296696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:17.297571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.298468 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.298747 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.299414 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:17.300232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:17.301086 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:17.307388 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 355/705] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:17.313090 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:17.351106 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:17.429742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a4b66-6506-46b4-833e-3cd2545f865e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:17.466871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:17.467524 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.467817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.468099 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.468860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] RBAC: Adding query filter params (interface=public, service_id=8ea16f1293a2441c88ebbaec23105a9d, region_id=RegionOne) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:17.469177 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:17.470590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d512969-ea96-431b-93ee-388329b7aa44 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:17.476506 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 351/706] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:17 2022] GET /identity/v3/endpoints?service_id=8ea16f1293a2441c88ebbaec23105a9d&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:17.484738 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:17.600504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac6e15-3a60-4dbd-bc53-639405d00250 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:17.638977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b89c0-4635-4551-a9dd-23563c935504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:17.639916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b89c0-4635-4551-a9dd-23563c935504 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.640406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b89c0-4635-4551-a9dd-23563c935504 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.640843 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b89c0-4635-4551-a9dd-23563c935504 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.641664 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b89c0-4635-4551-a9dd-23563c935504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:17.643207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b89c0-4635-4551-a9dd-23563c935504 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:17.650243 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 356/707] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:17 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:41:17.692522 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6f676-15af-4864-906c-ff168aeaf122 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.692812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6f676-15af-4864-906c-ff168aeaf122 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.693097 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6f676-15af-4864-906c-ff168aeaf122 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.694812 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 352/708] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:17.700620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:17.700926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:17.701171 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:17.709974 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:17.722918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:17.736017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:17.807326 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 357/709] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:18.069088 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6f676-15af-4864-906c-ff168aeaf122 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:18.170323 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:18.178774 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6f676-15af-4864-906c-ff168aeaf122 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:18.216162 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:18.217117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:18.217625 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:18.218099 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] PATH_INFO: `/v3/services/compute` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:18.218892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:18.221082 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:18.230401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:18.231597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:18.232639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:18.235146 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 353/710] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:18.243336 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:18.343931 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d801b01-cdb0-4c2e-9397-d672b23ed14c None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:18.379753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:18.380370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:18.380652 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:18.380929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:18.381567 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] RBAC: Adding query filter params (name=compute) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:18.381873 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:18.383335 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb42aee-bb75-444b-969f-82a3f8753ebe None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:18.384401 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:18.393100 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 358/711] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:18.404999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:18.503433 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cc413-9bae-416e-9c2e-6b726837475a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:18.539060 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:18.540090 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:18.540618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:18.541135 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:18.542015 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] RBAC: Adding query filter params (type=compute) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:18.542550 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:18.544258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a367e-099f-47fb-a3e5-9aba9d8aab44 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:18.550624 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 354/712] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:19.242735 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5776-961c-4833-bbcf-2de15510c0af None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.243284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5776-961c-4833-bbcf-2de15510c0af None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.243284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5776-961c-4833-bbcf-2de15510c0af None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.245251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 359/713] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:19.250831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b8df2-007e-465c-877f-f628c758405c None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.251284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b8df2-007e-465c-877f-f628c758405c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.251627 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b8df2-007e-465c-877f-f628c758405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.270521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-e43b8df2-007e-465c-877f-f628c758405c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:19.271859 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-e43b8df2-007e-465c-877f-f628c758405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:19.290448 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:19.348962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 355/714] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:19.442065 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0e9a-cba3-4d19-8a14-842e1d29f63b None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.442540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0e9a-cba3-4d19-8a14-842e1d29f63b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.442925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0e9a-cba3-4d19-8a14-842e1d29f63b None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.444783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 360/715] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:19.450229 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.450532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.450818 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.470100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:19.471380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:19.541806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 356/716] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:19.633310 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd0e9a-cba3-4d19-8a14-842e1d29f63b None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:19.748184 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd0e9a-cba3-4d19-8a14-842e1d29f63b None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:19.784949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:19.785978 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.786506 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.787021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.787870 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:19.789460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:19.802502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:19.803700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:19.804717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 361/717] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:19.807950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:19.830977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:19.897634 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:19.901347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:19.916715 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99f457-2957-49d7-9eab-b2ac8e8b4492 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:19.941206 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ef25-457d-40cc-85ca-e443862a7d80 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:19.955596 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:19.956217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.956510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.956795 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.957434 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:19.957751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:19.959159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37959f8-45a1-49ea-ac54-598ba8fc6e31 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:19.968875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 357/718] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:19.968875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfe41b-8631-4bf1-b365-7d47e06e093e None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.968875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfe41b-8631-4bf1-b365-7d47e06e093e None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.968875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfe41b-8631-4bf1-b365-7d47e06e093e None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.968875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 358/719] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:19.972841 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfe41b-8631-4bf1-b365-7d47e06e093e None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:19.978969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:19.980017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:19.980989 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:19.981269 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] PATH_INFO: `/v3/services/image` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:19.981866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:19.983229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-2316a026-63b6-4491-ad7c-5c3aa231744b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:19.993044 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:19.999207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:20.000574 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 362/720] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 05:41:19 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:20.001678 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.001678 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.001678 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.013827 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:20.015063 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:20.095332 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 363/721] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:20.095332 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:20.097495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfe41b-8631-4bf1-b365-7d47e06e093e None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.138502 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.139697 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.147855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.147855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.147855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:20.147855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.147855 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.154329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 359/722] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:41:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:20.156032 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:20.164591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:20.175835 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:20.221218 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9022260-cd31-47da-89e0-1f59026d2d1a None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.265456 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.266775 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.267245 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.267715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.268958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] RBAC: Adding query filter params (name=image) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:20.269448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.271560 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.278476 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 364/723] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:20.284923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:20.286608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076012f4-cd45-4778-b58d-ea8763f7f850 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.332036 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.333232 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.333730 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.334219 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.335117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.336881 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.402699 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164977d-bbe9-4485-8a09-49d76a9277cd None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.424545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:20.427509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 360/724] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:20.436675 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:20.461485 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.462930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.463436 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.463929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.465076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] RBAC: Adding query filter params (type=image) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:20.466125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.467920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.475929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 365/725] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:20.536113 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:20.581690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec1449-c290-41d3-bf0d-d1402cab4c28 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.633184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.633945 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.634240 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.634523 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.635149 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.636672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-2ee1fc1d-5909-45ae-aaf0-502f0c90bd84 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:20.647404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:20.648599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:20.649590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 361/726] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:20.654793 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:20.738746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21055d-c687-4349-9460-80cf69c9ea36 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.739076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21055d-c687-4349-9460-80cf69c9ea36 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.739341 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21055d-c687-4349-9460-80cf69c9ea36 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.741012 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 362/727] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:20.746196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.746485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.746768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.765606 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:20.766767 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:20.771992 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba172263-9680-4979-a762-c5d537ae8fdc None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.818720 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.819373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.819668 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.819949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.820593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:20.820935 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.822835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.830559 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 366/728] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:20.836194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:20.855169 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 363/729] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:20.890137 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:20.948010 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79fdd6-3249-4c63-b0a9-70e010691b6d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:20.983947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:20.984569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:20.984859 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:20.985228 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:20.985874 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:20.986186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:20.987584 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:20.993020 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 367/730] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:20.998727 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:21.068644 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:21.115867 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f4111-14ab-4d74-872e-dfbe784ea057 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:21.122016 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:21.153365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673b316-4dad-4e9a-b29d-809a335f7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:21.154385 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673b316-4dad-4e9a-b29d-809a335f7846 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:21.154902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673b316-4dad-4e9a-b29d-809a335f7846 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:21.155441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673b316-4dad-4e9a-b29d-809a335f7846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:21.156343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673b316-4dad-4e9a-b29d-809a335f7846 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:21.158007 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673b316-4dad-4e9a-b29d-809a335f7846 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:21.235545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c992-7081-48d1-8409-49ca09bfb86a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:21.236219 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 364/731] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:21.248597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:21.287242 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809d147-361f-4569-8c3a-789cb1df262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:21.288085 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809d147-361f-4569-8c3a-789cb1df262a None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:21.288338 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809d147-361f-4569-8c3a-789cb1df262a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:21.288616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809d147-361f-4569-8c3a-789cb1df262a None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:21.289363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809d147-361f-4569-8c3a-789cb1df262a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:21.290790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809d147-361f-4569-8c3a-789cb1df262a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:21.370259 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 368/732] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:21 2022] POST /identity/v3/services => generated 242 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:22.753554 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42d98-dfd3-43df-ae7d-8aa29a6808b4 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:22.754095 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42d98-dfd3-43df-ae7d-8aa29a6808b4 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:22.754095 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42d98-dfd3-43df-ae7d-8aa29a6808b4 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:22.755736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 365/733] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:22.761078 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:22.761526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:22.761699 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:22.772962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e5f3-00d9-4843-b973-8bc17b2b585a None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:22.773584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e5f3-00d9-4843-b973-8bc17b2b585a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:22.774074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e5f3-00d9-4843-b973-8bc17b2b585a None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:22.776034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 366/734] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:22.780513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:22.781047 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:22.781400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:22.781773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:22.782608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:22.801961 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:22.803358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:22.879083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 369/735] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:22.888339 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 367/736] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:23.156576 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:23.164381 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:23.222101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:23.255033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:23.268851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075a35-4cd7-44d6-b5a7-f94d5dc37297 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:23.293565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae8042-9a8b-4b44-94d5-e1d342c6e1f8 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:23.307415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:23.308242 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.308600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.308981 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.309713 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:23.311244 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:23.314167 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: WARNING keystone.server.flask.common [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 05:41:23.338304 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851f5dc-a786-4233-be06-368e0a8411b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:23.339256 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851f5dc-a786-4233-be06-368e0a8411b1 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.339749 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851f5dc-a786-4233-be06-368e0a8411b1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.340211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851f5dc-a786-4233-be06-368e0a8411b1 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.341073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851f5dc-a786-4233-be06-368e0a8411b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:23.342759 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851f5dc-a786-4233-be06-368e0a8411b1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:23.378878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.sql.core [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 05:41:23.378878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:41:23.378878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [parameters: {'id': '2c488554ac1c4867ac21d1c3218df738', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 05:41:23.378878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=153118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2c488554ac1c4867ac21d1c3218df738.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2c488554ac1c4867ac21d1c3218df738. Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:41:23.386930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result.read() Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:41:23.399607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application yield resource Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise self.value Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application session.commit() Oct 17 05:41:23.400855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:41:23.402248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:41:23.403318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result.read() Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:41:23.404327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [parameters: {'id': '2c488554ac1c4867ac21d1c3218df738', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:23.406158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:23.408983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2c488554ac1c4867ac21d1c3218df738. Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 370/737] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Oct 17 05:41:23 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:23.411592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:23.425773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 368/738] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:23 2022] POST /identity/v3/services => generated 242 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:23.497713 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e6633-6351-4446-95db-76825446f26c None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.498129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e6633-6351-4446-95db-76825446f26c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.498343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e6633-6351-4446-95db-76825446f26c None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.500038 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 369/739] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:23.505872 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb013-781c-40b6-97d0-74c3a1962e67 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.506185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb013-781c-40b6-97d0-74c3a1962e67 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.506459 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb013-781c-40b6-97d0-74c3a1962e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.527315 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-2ffdb013-781c-40b6-97d0-74c3a1962e67 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:23.528646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-2ffdb013-781c-40b6-97d0-74c3a1962e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:23.544946 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc35d5-eaa3-40ed-aa17-db95d7cc2a04 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:23.603524 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:23.604170 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.604462 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.604941 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.612868 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 370/740] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:23 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:23.613022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:23.614042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0e67f-87d8-48f6-bb0a-623041689db2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.614335 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0e67f-87d8-48f6-bb0a-623041689db2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.614611 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0e67f-87d8-48f6-bb0a-623041689db2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.614724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:23.619898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 371/741] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:23.621306 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.621605 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.621891 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:23.633562 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:23.634614 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:23.635718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 371/742] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:23.642722 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:23.649822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:23.649822 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:23.655392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:23.684514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:23.731644 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 372/743] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:23 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:23.770499 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c0a4b-f775-438e-9829-3e0e0ebb8028 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:23.808203 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:23.808821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:23.809121 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:23.809403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:23.810070 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:23.810383 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:23.811862 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:23.830850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 372/744] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:23.900479 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.013882 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ab0a-6560-46ca-8b48-ed01b9c07b88 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.050617 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.053476 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.054127 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.054413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.054690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.055284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.056707 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:24.066223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:24.067768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:24.068723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:24.070149 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 373/745] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:24.074646 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.118833 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.136844 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.188517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b692-69f5-4510-8e47-71053317aaab None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.206512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83039f26-358a-4ab1-9185-d4aa62913361 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.235887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.236544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.236832 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.237115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] PATH_INFO: `/v3/services/compute` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.237990 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.240131 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.250206 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.251150 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:24.251544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:24.252665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:24.258718 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 373/746] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:24.260188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.260188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.260188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:24.260188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.260188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.261934 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.264360 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 374/747] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:24.274968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.373411 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a761e65-bb60-41b1-98bb-456e4a89935a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.388923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.396301 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec635a4-29ad-4636-af18-4766b41dcefc None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.407113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.414701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.415737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.416257 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.416762 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.417636 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:24.418229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.419869 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.427728 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 374/748] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:24.433336 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.437015 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.437889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.438926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.439445 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.440276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:24.440820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.442440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.448189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 375/749] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:24.454307 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.560520 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eee4f-b12b-4dbf-bfdd-e37d18668ee2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.567991 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413b2f6-b1ed-4934-bbe2-6784d7c9f3f0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.603153 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.604177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.604684 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.605193 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.606054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:24.606593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.608400 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.612083 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.612731 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.613243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.613535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.614245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] RBAC: Adding query filter params (interface=admin, service_id=8ea16f1293a2441c88ebbaec23105a9d, region_id=RegionOne) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:24.614560 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.616001 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.616504 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 375/750] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:24.623467 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.631130 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 376/751] 10.222.0.13 () {56 vars in 1249 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/endpoints?service_id=8ea16f1293a2441c88ebbaec23105a9d&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:24.632267 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.665243 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.688778 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.738540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8057b6-627d-4a52-aa51-44a8f0c55aa7 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.757093 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b310ea0-c0e8-4399-9fad-0b04fd3d18b1 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.777314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.778608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.779099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.779591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.780615 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] RBAC: Adding query filter params (interface=public, service_id=7428073068ba44b6bac44156d2d58913, region_id=RegionOne) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:24.781128 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.783224 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.794176 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 376/752] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/endpoints?service_id=7428073068ba44b6bac44156d2d58913&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:24.795465 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:24.809064 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad613cf-2a4c-46d5-853d-58f220547c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.810370 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad613cf-2a4c-46d5-853d-58f220547c7f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.810899 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad613cf-2a4c-46d5-853d-58f220547c7f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.811383 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad613cf-2a4c-46d5-853d-58f220547c7f None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.812180 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad613cf-2a4c-46d5-853d-58f220547c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.814504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad613cf-2a4c-46d5-853d-58f220547c7f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.827975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 377/753] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services => generated 1957 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:41:24.922555 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56215961-f1af-4c1b-bc9c-f439000be9ad None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:24.942060 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:24.961362 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7385077-0ce7-4402-a418-cfd7e36fa338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:24.961985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385077-0ce7-4402-a418-cfd7e36fa338 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:24.962269 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385077-0ce7-4402-a418-cfd7e36fa338 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:24.962549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385077-0ce7-4402-a418-cfd7e36fa338 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:24.963169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7385077-0ce7-4402-a418-cfd7e36fa338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:24.964662 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7385077-0ce7-4402-a418-cfd7e36fa338 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:24.977120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 377/754] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:24 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:41:25.725510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260cce2-9e9d-4bb5-8c9d-6a6ec0822b2f None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:25.726081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260cce2-9e9d-4bb5-8c9d-6a6ec0822b2f None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:25.726081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260cce2-9e9d-4bb5-8c9d-6a6ec0822b2f None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:25.727702 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 378/755] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:25.733824 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45f354-ef53-4164-9c10-98d1b9926fdb None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:25.734159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45f354-ef53-4164-9c10-98d1b9926fdb None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:25.734401 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45f354-ef53-4164-9c10-98d1b9926fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:25.755699 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-ef45f354-ef53-4164-9c10-98d1b9926fdb None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:25.757633 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-ef45f354-ef53-4164-9c10-98d1b9926fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:25.838337 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 378/756] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:25 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:26.117693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260cce2-9e9d-4bb5-8c9d-6a6ec0822b2f None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.155916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be238e-a671-4d96-8c07-47706133c6c9 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.156276 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be238e-a671-4d96-8c07-47706133c6c9 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.156492 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be238e-a671-4d96-8c07-47706133c6c9 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.158024 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 379/757] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:26.163154 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.163491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.163777 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.189680 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:26.190875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:26.217683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:26.236097 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260cce2-9e9d-4bb5-8c9d-6a6ec0822b2f None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.278676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 380/758] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:26 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:26.278878 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:26.283773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.283773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.283773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] PATH_INFO: `/v3/services/image` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.283773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:26.283773 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:26.291712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:26.292909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:26.293975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 379/759] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 05:41:26 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:26.301456 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.345947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:26.403125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f4921-19cb-4a53-a5d4-d6084ca80da7 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.437811 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:26.438986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.439273 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.439589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.440236 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:26.440548 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:26.441992 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:26.447220 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 381/760] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:26.453075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.562368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15f833-4225-4906-93b3-7c4de7ac0d46 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.599918 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:26.600924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.601427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.601942 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.602786 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] RBAC: Adding query filter params (type=image) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:26.603389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:26.604989 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:26.609565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.611514 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 380/761] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:26.620753 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.638427 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:26.739131 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486dbc8-c297-4351-8960-9e07766696f2 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.746203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f8c1e-f164-47d4-af19-d6e9609425e5 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.747572 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:26.777353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:26.779093 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.779673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.780107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.781024 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:26.783237 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:26.789324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:26.790030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.790394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.790757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.791565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] RBAC: Adding query filter params (interface=public, service_id=b73ec788088347679955ffc15f8c4470, region_id=RegionOne) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:26.792063 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:26.793509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:26.805170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 381/762] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:41:26 2022] GET /identity/v3/endpoints?service_id=b73ec788088347679955ffc15f8c4470&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:26.806353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.836579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.sql.core [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 05:41:26.836579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 05:41:26.836579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [parameters: {'user_id': '3ea1c3234c9940b382efd2546023f07e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 05:41:26.836579 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=153118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result.read() Oct 17 05:41:26.847023 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:41:26.848489 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application yield resource Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise self.value Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application session.commit() Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:41:26.849589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:41:26.850601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:41:26.851732 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application result.read() Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:41:26.852743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [parameters: {'user_id': '3ea1c3234c9940b382efd2546023f07e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:26.855984 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 05:41:26.858052 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 382/763] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:41:26 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:26.861347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:26.904632 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:26.929081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055af3-42d7-40eb-8489-63f7c7481250 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.969593 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:26.970236 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:26.970527 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:26.970811 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:26.971604 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:26.972904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75688478-8918-4e97-9435-d047ee0c28aa None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:26.973121 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:26.987340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 382/764] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:41:26 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:41:27.021687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:27.022262 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.022554 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.022838 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.031797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:27.033521 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-0b57f182-e6b5-4ef7-adc6-9235d9cfccbb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:27.048311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:27.049507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:27.051080 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 383/765] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:41:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:27.056548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.096798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5486c-bd1a-49ba-84d8-5eb70f4ed78a None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.101949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5486c-bd1a-49ba-84d8-5eb70f4ed78a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.101949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5486c-bd1a-49ba-84d8-5eb70f4ed78a None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.106934 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 384/766] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:27.109006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d743d48-9ca2-4703-9993-319e2b4a94f9 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.109290 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d743d48-9ca2-4703-9993-319e2b4a94f9 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.109567 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d743d48-9ca2-4703-9993-319e2b4a94f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.133239 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-9d743d48-9ca2-4703-9993-319e2b4a94f9 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:27.135415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-9d743d48-9ca2-4703-9993-319e2b4a94f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:27.170357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:27.187697 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda585d-ea4a-422a-912e-bda746071fa9 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:27.225357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 385/767] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:27 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:27.230838 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5369fd2-840f-4a65-9d14-4a8eeffca0c5 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.231134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5369fd2-840f-4a65-9d14-4a8eeffca0c5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.231465 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5369fd2-840f-4a65-9d14-4a8eeffca0c5 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.232490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:27.233134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 386/768] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:27.233566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.234098 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.234617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.235581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:27.236097 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:27.237923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.238188 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:27.238314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.238483 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.259102 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:27.260366 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:27.280119 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 383/769] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:27.280711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:27.351641 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 387/770] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:27 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:27.511466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.623045 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8dd8e-9a98-41d8-b96c-ca63ada6c365 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:27.626048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.665026 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:27.665672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.665958 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.666239 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.666829 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:27.668384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:27.678203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:27.679391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:27.680409 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 384/771] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:27.685515 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.723415 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:27.736671 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590ad1e-e95b-430b-a992-7393d429f6b4 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:27.752213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:27.774346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:27.774994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.775284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.775593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.776189 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:27.777569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:27.786969 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:27.788234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:27.789213 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 388/772] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:27.794055 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.801131 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83e9d4-d04e-4eb5-97e2-a5a3811b56fd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:27.840712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:27.841331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.841621 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.841905 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.842539 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:27.843117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:27.844278 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:27.854040 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 385/773] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:27.855024 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.909514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9aee-3b2c-4aa7-94a9-84671110c1d0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:27.946163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:27.946887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:27.947179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:27.947507 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:27.948153 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] RBAC: Adding query filter params (name=compute) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:27.948468 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:27.949879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:27.955145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 389/774] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:27.961201 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:27.967945 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9eb62-ffb2-4e8e-95ab-c7e8fc488163 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:27.981648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:27.998852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:28.007412 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:28.008031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:28.008324 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:28.008597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:28.009234 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:28.009549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:28.010937 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:28.021005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 386/775] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:28.021767 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:28.070423 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58b31e-7066-4a13-885b-16c731f7301a None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:28.111983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:28.112602 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:28.112894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:28.113449 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:28.113825 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] RBAC: Adding query filter params (type=compute) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:28.114166 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:28.115542 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:28.122365 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 390/776] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 05:41:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:28.127891 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:28.129170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747e034-5950-42f9-af50-1b3ed1a60c46 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:28.166355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419d6b5-ccf4-4cac-a47d-6d2061402b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:28.166953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419d6b5-ccf4-4cac-a47d-6d2061402b0c None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:28.167276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419d6b5-ccf4-4cac-a47d-6d2061402b0c None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:28.167608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419d6b5-ccf4-4cac-a47d-6d2061402b0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:28.168308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419d6b5-ccf4-4cac-a47d-6d2061402b0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:28.169725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419d6b5-ccf4-4cac-a47d-6d2061402b0c None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:28.223051 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:28.242636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 387/777] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:28.244947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27cf09-a2cc-400e-915d-c873dafaf696 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:28.290981 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e6b6c-dd04-4d37-8356-5468064cbcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:28.291635 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e6b6c-dd04-4d37-8356-5468064cbcf3 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:28.291920 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e6b6c-dd04-4d37-8356-5468064cbcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:28.292194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e6b6c-dd04-4d37-8356-5468064cbcf3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:28.292876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e6b6c-dd04-4d37-8356-5468064cbcf3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:28.294287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e6b6c-dd04-4d37-8356-5468064cbcf3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:28.415245 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 391/778] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:41:29.246483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b55d-896e-4646-a68b-03d1a9c9626a None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:29.247072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b55d-896e-4646-a68b-03d1a9c9626a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:29.247072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b55d-896e-4646-a68b-03d1a9c9626a None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:29.248548 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 388/779] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:29.253580 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:29.253976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:29.254145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:29.272060 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:29.273247 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:29.347460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 392/780] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:29.610941 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b55d-896e-4646-a68b-03d1a9c9626a None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:29.666643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:29.716176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b55d-896e-4646-a68b-03d1a9c9626a None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:29.751123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:29.751783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:29.752074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:29.752354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] PATH_INFO: `/v3/services/image` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:29.752947 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:29.754331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:29.763173 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:29.764483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:29.765568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 389/781] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 05:41:29 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:29.773190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:29.841914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:29.883544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a12c4-487f-4b78-98a8-d1fa62c833e8 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:29.917784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:29.918406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:29.918688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:29.918962 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:29.919623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] RBAC: Adding query filter params (name=image) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:29.919932 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:29.921294 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:29.927460 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 393/782] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:29.932290 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:30.037935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47638a3a-b3a2-4e92-8249-9a57b0b3ef92 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:30.055410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:30.074113 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:30.074870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.075283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.075692 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.076442 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] RBAC: Adding query filter params (type=image) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:30.076856 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:30.078327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8bf69-9f00-4f86-b36d-87d7159eec1f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:30.088628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 390/783] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:30.089861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:30.193393 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc0967-c9d7-4b47-87c0-36afda7fc0a0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:30.216066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:30.228317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fce220-9869-4832-b923-f697f3631284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:30.229007 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fce220-9869-4832-b923-f697f3631284 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.229360 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fce220-9869-4832-b923-f697f3631284 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.229708 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fce220-9869-4832-b923-f697f3631284 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.230446 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fce220-9869-4832-b923-f697f3631284 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:30.231913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fce220-9869-4832-b923-f697f3631284 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:30.335338 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 394/784] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:41:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:30.447654 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843e5a-377c-4db8-ac18-3929f15a53d0 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.448195 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843e5a-377c-4db8-ac18-3929f15a53d0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.448195 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843e5a-377c-4db8-ac18-3929f15a53d0 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.449636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 391/785] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:30.456013 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5634-4338-4c9c-ba46-a66b0c18ef76 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.456339 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5634-4338-4c9c-ba46-a66b0c18ef76 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.456591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5634-4338-4c9c-ba46-a66b0c18ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.475976 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-c23c5634-4338-4c9c-ba46-a66b0c18ef76 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:30.477218 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-c23c5634-4338-4c9c-ba46-a66b0c18ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:30.560947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 395/786] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:30.650565 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192c83-5e8d-45e4-a50d-04dc33d39d79 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.650891 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192c83-5e8d-45e4-a50d-04dc33d39d79 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.651132 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192c83-5e8d-45e4-a50d-04dc33d39d79 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.652629 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 392/787] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:30.657724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.658018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.658303 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.676613 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:30.677799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:30.745983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 396/788] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:30.839046 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b192c83-5e8d-45e4-a50d-04dc33d39d79 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:30.944360 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b192c83-5e8d-45e4-a50d-04dc33d39d79 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:30.977549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:30.978305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:30.978689 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:30.979061 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:30.980318 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:30.981831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:30.983775 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 393/789] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:30.989979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.015866 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.028964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:31.072927 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:31.105862 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d2a49-0697-472d-8b19-130c036b26ff None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:31.126737 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994caf-0627-454e-bae3-2dd4c240a82f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:31.148590 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:31.149215 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:31.149654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:31.149956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:31.150608 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:31.152143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:31.173065 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:31.173860 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:31.174143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:31.174421 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:31.186589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:31.192691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:31.194885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 394/790] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:41:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:31.210576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.316761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643c0e8-38df-4758-a527-686855edce76 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:31.316992 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 397/791] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:30 2022] POST /identity/v3/projects => generated 311 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:31.318028 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:31.347598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:31.357520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:31.357520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:31.357520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:31.357520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:31.357520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:31.357520 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:31.365606 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 395/792] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:41:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:31.368712 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.466726 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:31.484655 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce4a76-3310-4250-80c5-c67529bb1670 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:31.521592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:31.522233 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:31.522508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:31.522784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:31.533644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:31.535090 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:31.549530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:31.550893 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:31.552342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:31.552342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:31.552342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:31.552342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 398/793] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:31.556611 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.661089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e660bc-949c-4326-90da-8e22a725cea0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:31.696081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:31.696697 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:31.696980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:31.697258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:31.697904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:31.698210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:31.699614 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:31.711813 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 396/794] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:31.717392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.822580 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb143933-d20d-4251-801d-e83833d20021 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:31.858423 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:31.860979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:31.860979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:31.860979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:31.868034 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:31.869475 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:31.871373 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:31.881921 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:31.885018 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:31.886200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 399/795] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:41:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:31.888823 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:31.920400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:31.992402 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265df7b9-4581-4ad8-a3f0-9fc531eea1d8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:32.026080 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:32.026695 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.026981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.027260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.028016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:32.028331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:32.029768 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:32.053000 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 397/796] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:41:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:32.054659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:32.158397 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc40085-45fb-41d6-b68b-7803da651a6f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:32.192734 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:32.193282 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.193585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.193861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.204414 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:32.205885 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:32.220350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:32.221444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:32.222494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:32.222494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:32.222494 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:32.226212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 400/797] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:32.227677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:32.321449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:32.334390 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f686b7-e65b-49f0-ac0a-cf030f777080 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:32.368966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:32.369581 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.369867 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.370144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.370787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:32.371096 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:32.372501 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:32.384072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 398/798] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:32.389599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:32.442323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:32.498074 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da0f94-1234-4ff7-845c-db65f96578ce None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:32.532422 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:32.533054 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.533342 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.533623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.540389 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:32.541836 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-b25e792d-4cce-49ef-97f6-61c6c87e4806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:32.553454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:32.554484 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:32.555977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:32.556827 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 401/799] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:32.560629 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:32.561648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3970a75-6dab-4e55-9a79-aa03ab0874bc None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.561939 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3970a75-6dab-4e55-9a79-aa03ab0874bc None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.562217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3970a75-6dab-4e55-9a79-aa03ab0874bc None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.563654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 402/800] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:41:32.568648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.569134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.569596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.589187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:32.590626 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:32.666570 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf128b3-f694-4869-afec-7fa0df38e68d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:32.667547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 403/801] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:32.706174 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:32.706814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.707098 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.707497 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.708192 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:32.708503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:32.709932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:32.727290 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 399/802] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:32.728168 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:32.771272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:32.835327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444c386-e88c-4a20-96a8-17dc77f81a79 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:32.869848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:32.870502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:32.870786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:32.871066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/921c6749f51c49d78748876ec65bd71f/roles/d5f3d562e3ac4f54b99a938c6e104cd4` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:32.873234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=921c6749f51c49d78748876ec65bd71f, role_id=d5f3d562e3ac4f54b99a938c6e104cd4)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:32.874677 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:32.933194 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 404/803] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:41:32 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/921c6749f51c49d78748876ec65bd71f/roles/d5f3d562e3ac4f54b99a938c6e104cd4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:41:32.943608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:32.956699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:33.057717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6c23-7210-4257-b1f2-e5b11f484c07 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:33.096584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:33.103455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:33.103455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:33.103455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:33.103455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:33.103455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:33.144309 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.sql.core [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:41:33.144309 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:41:33.144309 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [parameters: {'id': '9d53b50bbcd94023be3b16b8f4e9dfe0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:41:33.144309 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=153119) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:41:33.151761 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result.read() Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:41:33.152766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application yield resource Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise self.value Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application session.commit() Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:41:33.153976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:41:33.155140 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:41:33.157690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application result.read() Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:41:33.159858 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [parameters: {'id': '9d53b50bbcd94023be3b16b8f4e9dfe0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:41:33.162423 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:33.164555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:33.166915 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 400/804] 10.222.0.13 () {60 vars in 1137 bytes} [Mon Oct 17 05:41:32 2022] POST /identity/v3/domains => generated 190 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:33.169038 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:33.266854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:33.270124 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188d62d-e72d-48f1-ac92-d5be86900b73 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:33.306348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:33.307078 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:33.307479 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:33.307884 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:33.318027 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:33.319620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d1a26d0e-a08e-414e-8825-bed2a9d563e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:33.335076 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:33.336386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:33.337630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:33.337630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:33.337630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:33.342696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 405/805] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:33.344615 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:33.417014 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:33.450956 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aab48-e850-4b36-97ad-42f116f87cdd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:33.489673 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:33.490305 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:33.490592 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:33.490869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:33.491551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:33.491862 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:33.493237 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625cefc-1064-41b0-b2de-a7987e170211 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:33.505692 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 401/806] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:33.581155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892413-f6c3-491d-910a-eeed6e861f30 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:33.581676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892413-f6c3-491d-910a-eeed6e861f30 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:33.581676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892413-f6c3-491d-910a-eeed6e861f30 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:33.583234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 406/807] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:33.588868 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:33.589390 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:33.589481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:33.609331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:33.610604 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:33.689835 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 402/808] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:33.959549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892413-f6c3-491d-910a-eeed6e861f30 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:34.058936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892413-f6c3-491d-910a-eeed6e861f30 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:34.092804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fb887-f2db-43b3-b68f-b17469ef04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:34.099448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb887-f2db-43b3-b68f-b17469ef04a0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:34.099448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb887-f2db-43b3-b68f-b17469ef04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:34.099448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb887-f2db-43b3-b68f-b17469ef04a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:34.099448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb887-f2db-43b3-b68f-b17469ef04a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:34.099448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb887-f2db-43b3-b68f-b17469ef04a0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:34.099448 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 407/809] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:34.104069 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:34.166359 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:34.206061 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50b3fd-5a3a-49ad-b1bb-efae99a1253e None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:34.239800 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029f90-b279-4738-8e5c-608dfe963849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:34.240343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029f90-b279-4738-8e5c-608dfe963849 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:34.240650 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029f90-b279-4738-8e5c-608dfe963849 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:34.240913 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029f90-b279-4738-8e5c-608dfe963849 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:34.241568 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029f90-b279-4738-8e5c-608dfe963849 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:34.242990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029f90-b279-4738-8e5c-608dfe963849 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:34.332522 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 403/810] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:41:34 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:35.085745 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c667a-2bd4-4877-950b-3bce98ae112a None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:35.086266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c667a-2bd4-4877-950b-3bce98ae112a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:35.086266 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c667a-2bd4-4877-950b-3bce98ae112a None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:35.087603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 408/811] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:35.093101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:35.093544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:35.093658 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:35.112308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:35.113489 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:35.180820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 404/812] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:35.448916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c667a-2bd4-4877-950b-3bce98ae112a None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:35.471509 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:35.551254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c667a-2bd4-4877-950b-3bce98ae112a None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:35.584277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:35.584907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:35.585195 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:35.585478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:35.596630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:35.601416 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:35.602644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:35.603695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 409/813] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 05:41:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:35.608556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:35.657625 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:35.711388 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d7d35-0b94-4e0d-a884-fe65b20ed491 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:35.742102 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:35.742710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:35.742997 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:35.743275 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:35.743965 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:35.744273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:35.745637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:35.753886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 405/814] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 05:41:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:35.759790 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:35.842593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:35.861444 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982957d-5ba5-4060-9a17-4041344d9f5c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:35.895095 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:35.895754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:35.896051 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:35.896331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:35.907160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:35.908652 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:35.923363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:35.924600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:35.925657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:35.925657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:35.925657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:35.925657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 410/815] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:35.930240 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.030358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573af3e-4e09-4620-97e2-2c2d3fd46d7a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:36.040468 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:36.064621 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:36.065238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.065524 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.065803 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.066452 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:36.066767 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:36.068176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:36.080441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 406/816] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:36.086092 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.187876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d31b4-96f0-4511-8319-ac34eec83bfb None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:36.216084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:36.221567 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:36.222196 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.222485 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.222762 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.230064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:36.231544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:41:36.243158 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:36.244430 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:36.245525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 411/817] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 05:41:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:36.250182 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.348543 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce94ba-1ccd-4f93-9dcd-6fc3e1e23910 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:36.381970 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:36.382571 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.382851 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.383126 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.383874 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:36.384184 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:36.385595 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c913a-131e-49f2-b624-a93357e89488 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:36.402772 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 407/818] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 05:41:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:36.408797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.510376 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b575ac-1688-40b1-8878-d739240a1eb0 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:36.533576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f10777-ffb9-466d-8507-2f3b038b55d6 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.533871 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f10777-ffb9-466d-8507-2f3b038b55d6 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.534161 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f10777-ffb9-466d-8507-2f3b038b55d6 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.535555 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 408/819] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:36.540384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.540670 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.540950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.544644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:36.545248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.545530 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.545805 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.561150 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:36.562365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:36.585437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:36.604149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:36.613877 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:36.615375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:36.630658 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 409/820] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:36.712481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 412/821] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:41:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:41:36.718277 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.817776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db268a-1ac1-47f7-a799-7891460f6f75 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:36.850441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:36.851073 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:36.851389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:36.851672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:36.861943 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:36.863410 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:41:36.878202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:36.879453 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:41:36.880692 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:41:36.880692 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:41:36.880692 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:36.883991 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 410/822] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:36.885161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.899883 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:36.912949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:36.974683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:36.983355 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6837e2-66d1-43f0-8ede-97cfdaaaaacc None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.000975 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337da5d-1764-4d2a-a8e4-eb3fe92ddb82 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.018337 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.018945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.019232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.019541 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.020191 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:37.020496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.021877 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.033534 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 413/823] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:37.034606 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.035346 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.035704 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.036048 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.039107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.046681 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:37.052044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:37.053114 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:37.054608 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 411/824] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:41:37.058908 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.129004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:37.139200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8cde-8fb0-4acd-ae5b-10d9ab224871 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.159234 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422129f-b28f-4617-9b81-dc55b73721dd None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.172985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.173688 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.174017 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.174336 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.180902 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.182343 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.192958 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:41:37.193526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:37.194508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:41:37.195627 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:37.196123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.196123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.196123 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.196352 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 414/825] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:37.196440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:37.196440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.196440 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.200553 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.204633 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 412/826] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:41:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:37.210204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.215363 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:37.300741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca883af-c204-4d19-9f09-a664beac64ce None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.310549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f1a7-8cb7-4b64-927a-b1d4903fe483 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.333682 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.334287 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.334578 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.334861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.335596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:37.335913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.337357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.339104 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:37.344647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.345271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.345553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.345828 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] PATH_INFO: `/v3/users/5ed332ac18b945caa82e5ca66c060d8e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.350045 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 415/827] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:37.356164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.359844 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] RBAC: Authorizing `identity:get_user(user_id=5ed332ac18b945caa82e5ca66c060d8e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.361331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.362968 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 413/828] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:37 2022] GET /identity/v3/users/5ed332ac18b945caa82e5ca66c060d8e => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:37.368637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.426814 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:37.455473 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ab6f-3a83-454d-909c-5fdaec76f5ad None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.467418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e310be1-dfd2-475a-89c3-2a23821d460d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.490182 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.490787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.491072 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.491378 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.492141 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] RBAC: Adding query filter params (role.id=d5f3d562e3ac4f54b99a938c6e104cd4, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=921c6749f51c49d78748876ec65bd71f) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:37.492454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.493888 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.499691 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 416/829] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:37 2022] GET /identity/v3/role_assignments?user.id=921c6749f51c49d78748876ec65bd71f&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=d5f3d562e3ac4f54b99a938c6e104cd4 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:37.502468 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2679-d975-4576-b64a-169ce725401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.503110 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2679-d975-4576-b64a-169ce725401e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.503430 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2679-d975-4576-b64a-169ce725401e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.503714 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2679-d975-4576-b64a-169ce725401e None admin] PATH_INFO: `/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.504885 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff2679-d975-4576-b64a-169ce725401e None admin] RBAC: Authorizing `identity:get_project(project_id=ada76ebc70414ef68a2e4fc9a2bbb8d9)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.506378 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff2679-d975-4576-b64a-169ce725401e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.508127 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 414/830] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:41:37 2022] GET /identity/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:37.514152 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:37.621275 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b23e9-15f2-4097-9c19-89ce990b71db None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:37.658256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c69a2-0c9e-485f-8a3e-eccb5d99c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:37.658947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c69a2-0c9e-485f-8a3e-eccb5d99c346 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:37.659234 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c69a2-0c9e-485f-8a3e-eccb5d99c346 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:37.659547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c69a2-0c9e-485f-8a3e-eccb5d99c346 None admin] PATH_INFO: `/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9/users/5ed332ac18b945caa82e5ca66c060d8e/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:37.661539 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c69a2-0c9e-485f-8a3e-eccb5d99c346 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada76ebc70414ef68a2e4fc9a2bbb8d9, user_id=5ed332ac18b945caa82e5ca66c060d8e, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:37.663006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c69a2-0c9e-485f-8a3e-eccb5d99c346 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:37.666418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:37.750129 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 417/831] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:41:37 2022] PUT /identity/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9/users/5ed332ac18b945caa82e5ca66c060d8e/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:41:39.872419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa765e-6d6f-4a79-9490-70d8e3ade237 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:39.872921 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa765e-6d6f-4a79-9490-70d8e3ade237 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:39.873053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa765e-6d6f-4a79-9490-70d8e3ade237 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:39.874313 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 415/832] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:39.880272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:39.880715 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:39.880827 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:39.898687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:39.899885 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:39.965673 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 418/833] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:40.234916 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa765e-6d6f-4a79-9490-70d8e3ade237 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:40.294626 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:40.335922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa765e-6d6f-4a79-9490-70d8e3ade237 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:40.369437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:40.370078 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:40.370368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:40.370649 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:40.380621 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:40.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:40.386636 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:41:40.387922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 416/834] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:40.392958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:40.488949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204bd14-2e3e-4ced-9941-0e64acdf885b None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:40.513153 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:40.522099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:40.525597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:40.525597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:40.525597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:40.525597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:40.525597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:40.525597 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:40.533940 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 419/835] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:40.539825 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:40.638696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94624470-4ac1-48ab-99f7-255d959555e3 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:40.656063 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:40.672828 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:40.674028 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:40.674028 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:40.674028 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] PATH_INFO: `/v3/users/5ed332ac18b945caa82e5ca66c060d8e` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:40.675336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] RBAC: Authorizing `identity:get_user(user_id=5ed332ac18b945caa82e5ca66c060d8e)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:40.676779 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:40.678321 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 417/836] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:40 2022] GET /identity/v3/users/5ed332ac18b945caa82e5ca66c060d8e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:40.684083 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:40.780648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e8c1-a3b2-41e0-9d7b-9393f4319cb3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:40.815657 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:40.816271 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:40.816559 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:40.816838 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:40.870452 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:40.882842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:40.884318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:40.979796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 420/837] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:41:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:41:40.986302 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:41.084357 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3ac77-77a6-4a60-bc1b-a7b1e89fee3f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:41.231244 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67d00a-fb30-4615-a91a-211a66cf2c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:41.231891 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67d00a-fb30-4615-a91a-211a66cf2c5e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:41.232177 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67d00a-fb30-4615-a91a-211a66cf2c5e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:41.232459 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67d00a-fb30-4615-a91a-211a66cf2c5e None admin] PATH_INFO: `/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:41.233566 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67d00a-fb30-4615-a91a-211a66cf2c5e None admin] RBAC: Authorizing `identity:get_project(project_id=ada76ebc70414ef68a2e4fc9a2bbb8d9)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:41.235609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67d00a-fb30-4615-a91a-211a66cf2c5e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:41.237211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 418/838] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:41:40 2022] GET /identity/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9 => generated 311 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:41.243816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:41.329799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:41.341427 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee961c-3aea-4a07-85a4-9aae6b9661d8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:41.375340 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:41.375958 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:41.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:41.376519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:41.377238 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=ada76ebc70414ef68a2e4fc9a2bbb8d9, user.id=5ed332ac18b945caa82e5ca66c060d8e) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:41.377549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:41.378983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275ebfa-75c0-44fb-9090-4a48f821254a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:41.384665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 421/839] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:41 2022] GET /identity/v3/role_assignments?user.id=5ed332ac18b945caa82e5ca66c060d8e&scope.project.id=ada76ebc70414ef68a2e4fc9a2bbb8d9&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:43.511687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb9ccb-422e-4b4a-8d83-91541d65e42d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:43.512260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb9ccb-422e-4b4a-8d83-91541d65e42d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:43.512260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb9ccb-422e-4b4a-8d83-91541d65e42d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:43.513587 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 419/840] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:43.519344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:43.519782 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:43.519918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:43.539135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:43.540348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:43.605066 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 422/841] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:43.876391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb9ccb-422e-4b4a-8d83-91541d65e42d None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:43.931749 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:43.976275 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb9ccb-422e-4b4a-8d83-91541d65e42d None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:44.008723 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab01571-6dc5-48c5-ae21-315d9b2cfaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:44.009338 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab01571-6dc5-48c5-ae21-315d9b2cfaed None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:44.009623 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab01571-6dc5-48c5-ae21-315d9b2cfaed None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:44.009903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab01571-6dc5-48c5-ae21-315d9b2cfaed None admin] PATH_INFO: `/v3/domains/default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:44.011016 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab01571-6dc5-48c5-ae21-315d9b2cfaed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:44.012474 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab01571-6dc5-48c5-ae21-315d9b2cfaed None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:44.013994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 420/842] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:41:44.020372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:44.099372 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:44.117796 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b624a-6c4b-4190-be31-83d0542b004c None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:44.152042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e883fc-86d9-4a42-8d53-cf4f379ab1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:44.152654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e883fc-86d9-4a42-8d53-cf4f379ab1f6 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:44.153056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e883fc-86d9-4a42-8d53-cf4f379ab1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:44.153344 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e883fc-86d9-4a42-8d53-cf4f379ab1f6 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:44.154003 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e883fc-86d9-4a42-8d53-cf4f379ab1f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:44.155535 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e883fc-86d9-4a42-8d53-cf4f379ab1f6 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:44.241845 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 423/843] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:41:44 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:46.378144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5f6e6-174a-462d-8fa9-6d26521e0660 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:46.378701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5f6e6-174a-462d-8fa9-6d26521e0660 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:46.378701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5f6e6-174a-462d-8fa9-6d26521e0660 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:46.380053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 421/844] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:46.385270 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:46.385697 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:46.385841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:46.404383 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:46.405576 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:46.471035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 424/845] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:46.735602 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5f6e6-174a-462d-8fa9-6d26521e0660 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:46.835250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5f6e6-174a-462d-8fa9-6d26521e0660 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:46.856721 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:46.867693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:46.868551 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:46.868701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:46.868854 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] PATH_INFO: `/v3/roles/c040847a15b148f893d961a9a578db29` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:46.870228 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] RBAC: Authorizing `identity:get_role(role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:46.871609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:46.872893 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 422/846] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:46 2022] GET /identity/v3/roles/c040847a15b148f893d961a9a578db29 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:46.879171 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:46.944765 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:46.976907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714fd0-fe8d-44a5-b117-6cf03d7445c5 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:47.011816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:47.011816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:47.011816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:47.012170 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] PATH_INFO: `/v3/users/904bd34de4f1459ea5ba71eea81a70e7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:47.024704 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] RBAC: Authorizing `identity:get_user(user_id=904bd34de4f1459ea5ba71eea81a70e7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:47.026144 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:47.027774 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 425/847] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:46 2022] GET /identity/v3/users/904bd34de4f1459ea5ba71eea81a70e7 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:47.033575 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:47.131557 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d67bc-2702-4bbc-bea2-b6b61dc33007 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:47.163894 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8452281-33ee-45ec-9b81-9b3c6c80c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:47.164512 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8452281-33ee-45ec-9b81-9b3c6c80c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:47.164801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8452281-33ee-45ec-9b81-9b3c6c80c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:47.165081 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8452281-33ee-45ec-9b81-9b3c6c80c5e6 None admin] PATH_INFO: `/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:47.166313 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8452281-33ee-45ec-9b81-9b3c6c80c5e6 None admin] RBAC: Authorizing `identity:get_project(project_id=ada76ebc70414ef68a2e4fc9a2bbb8d9)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:47.167782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8452281-33ee-45ec-9b81-9b3c6c80c5e6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:47.169405 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 423/848] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:41:47 2022] GET /identity/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:47.176073 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:47.273332 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc92c00-3787-4f88-bb76-daf77a5b487f None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:47.298979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:47.307519 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212eb1b-b9bd-4b9c-a768-ff17969b3632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:47.308169 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212eb1b-b9bd-4b9c-a768-ff17969b3632 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:47.308454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212eb1b-b9bd-4b9c-a768-ff17969b3632 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:47.308731 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212eb1b-b9bd-4b9c-a768-ff17969b3632 None admin] PATH_INFO: `/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9/users/904bd34de4f1459ea5ba71eea81a70e7/roles/c040847a15b148f893d961a9a578db29` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:47.310843 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212eb1b-b9bd-4b9c-a768-ff17969b3632 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada76ebc70414ef68a2e4fc9a2bbb8d9, user_id=904bd34de4f1459ea5ba71eea81a70e7, role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:47.312916 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212eb1b-b9bd-4b9c-a768-ff17969b3632 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:47.389159 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 426/849] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:41:47 2022] PUT /identity/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9/users/904bd34de4f1459ea5ba71eea81a70e7/roles/c040847a15b148f893d961a9a578db29 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:41:49.481904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d289e8f-aac3-4ce6-bc52-47e8806da105 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:49.482461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d289e8f-aac3-4ce6-bc52-47e8806da105 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:49.482461 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d289e8f-aac3-4ce6-bc52-47e8806da105 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:49.484450 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 424/850] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:49.489761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:49.490185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:49.490321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:49.509019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:49.510222 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:49.575952 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 427/851] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:49.845205 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d289e8f-aac3-4ce6-bc52-47e8806da105 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:49.904245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:49.946742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d289e8f-aac3-4ce6-bc52-47e8806da105 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:49.980447 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:49.981074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:49.981363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:49.981649 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] PATH_INFO: `/v3/roles/c040847a15b148f893d961a9a578db29` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:49.983033 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] RBAC: Authorizing `identity:get_role(role_id=c040847a15b148f893d961a9a578db29)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:49.984530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:49.985834 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 425/852] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:49 2022] GET /identity/v3/roles/c040847a15b148f893d961a9a578db29 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:49.991912 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:50.205867 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056152ba-6593-4c4f-8b7f-3ba13c231d70 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:50.221583 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:50.238898 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:50.239572 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:50.239868 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:50.240146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] PATH_INFO: `/v3/users/904bd34de4f1459ea5ba71eea81a70e7` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:50.241296 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] RBAC: Authorizing `identity:get_user(user_id=904bd34de4f1459ea5ba71eea81a70e7)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:50.242812 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:50.244417 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 428/853] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:49 2022] GET /identity/v3/users/904bd34de4f1459ea5ba71eea81a70e7 => generated 286 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:50.250828 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:50.347044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88198edc-2a11-4516-a8cd-2df3ac0b5f6e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:50.371553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:50.380418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:50.381021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:50.381306 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:50.381582 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:50.447282 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:50.448746 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:50.543274 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 426/854] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:41:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:41:50.549940 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:50.644529 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf49cb2-9dab-457f-856f-26a46bf27c04 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:50.676964 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5a0ed-73aa-4cb5-9b12-fdcea2ae1f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:50.677576 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a0ed-73aa-4cb5-9b12-fdcea2ae1f7b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:50.677858 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a0ed-73aa-4cb5-9b12-fdcea2ae1f7b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:50.678134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a0ed-73aa-4cb5-9b12-fdcea2ae1f7b None admin] PATH_INFO: `/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:50.679240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a0ed-73aa-4cb5-9b12-fdcea2ae1f7b None admin] RBAC: Authorizing `identity:get_project(project_id=ada76ebc70414ef68a2e4fc9a2bbb8d9)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:50.680695 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a0ed-73aa-4cb5-9b12-fdcea2ae1f7b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:50.682283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 429/855] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:41:50 2022] GET /identity/v3/projects/ada76ebc70414ef68a2e4fc9a2bbb8d9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:50.688839 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:50.715079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:50.786448 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b77d7-a071-4988-a92d-752c942e4c52 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:50.820053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:50.820682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:50.820965 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:50.821239 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:50.821962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] RBAC: Adding query filter params (role.id=c040847a15b148f893d961a9a578db29, scope.project.id=ada76ebc70414ef68a2e4fc9a2bbb8d9, user.id=904bd34de4f1459ea5ba71eea81a70e7) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:50.822275 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:50.823726 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077356-69ff-43d6-9dad-c987bf982f40 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:50.829926 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 427/856] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:41:50 2022] GET /identity/v3/role_assignments?user.id=904bd34de4f1459ea5ba71eea81a70e7&scope.project.id=ada76ebc70414ef68a2e4fc9a2bbb8d9&role.id=c040847a15b148f893d961a9a578db29 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:52.963186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d4850-7d0c-4520-b828-3d6c987781ce None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:52.963813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d4850-7d0c-4520-b828-3d6c987781ce None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:52.963813 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d4850-7d0c-4520-b828-3d6c987781ce None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:52.965118 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 430/857] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:52.970974 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:52.971412 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:52.971536 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:52.990516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:52.991736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:53.055722 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 428/858] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 05:41:53.323420 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d4850-7d0c-4520-b828-3d6c987781ce None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:53.376698 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:53.423341 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d4850-7d0c-4520-b828-3d6c987781ce None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:53.456937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c59c0-068a-4d37-ba68-bad6924484bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:53.457554 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c59c0-068a-4d37-ba68-bad6924484bc None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:53.457840 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c59c0-068a-4d37-ba68-bad6924484bc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:53.458119 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c59c0-068a-4d37-ba68-bad6924484bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:53.459232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c59c0-068a-4d37-ba68-bad6924484bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:53.460689 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c59c0-068a-4d37-ba68-bad6924484bc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:53.462232 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 431/859] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:53.468019 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:53.488503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:53.564380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4a8f2-57ad-4432-b708-621268d60436 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:53.597252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646113f-004b-47b6-bf8a-2974eca82bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:53.597852 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646113f-004b-47b6-bf8a-2974eca82bfb None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:53.598138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646113f-004b-47b6-bf8a-2974eca82bfb None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:53.598416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646113f-004b-47b6-bf8a-2974eca82bfb None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:53.599079 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646113f-004b-47b6-bf8a-2974eca82bfb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:53.600539 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646113f-004b-47b6-bf8a-2974eca82bfb None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:53.670085 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 429/860] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:41:53 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:55.851974 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f5724-caa4-4afe-b39d-cb0bcf4409d4 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:55.852497 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f5724-caa4-4afe-b39d-cb0bcf4409d4 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:55.852599 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f5724-caa4-4afe-b39d-cb0bcf4409d4 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:55.853936 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 432/861] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:55.859832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:55.860280 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:55.860390 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:55.878180 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:55.879401 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:55.944385 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 430/862] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:56.231647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f5724-caa4-4afe-b39d-cb0bcf4409d4 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:56.331563 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f5724-caa4-4afe-b39d-cb0bcf4409d4 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:56.350886 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:56.364739 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63808fad-4443-4c21-b70b-0f5220c91d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:56.365355 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63808fad-4443-4c21-b70b-0f5220c91d12 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:56.365682 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63808fad-4443-4c21-b70b-0f5220c91d12 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:56.366008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63808fad-4443-4c21-b70b-0f5220c91d12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:56.367108 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63808fad-4443-4c21-b70b-0f5220c91d12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:56.368609 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63808fad-4443-4c21-b70b-0f5220c91d12 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:56.370242 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 433/863] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:41:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:56.376616 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:56.378839 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:56.475356 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0fe9a-f2bd-48d6-af54-3263e2b65872 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:56.509509 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad10f9-63f2-4370-8d88-daa01b90ee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:56.510055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad10f9-63f2-4370-8d88-daa01b90ee25 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:56.510346 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad10f9-63f2-4370-8d88-daa01b90ee25 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:56.510626 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad10f9-63f2-4370-8d88-daa01b90ee25 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:56.511283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad10f9-63f2-4370-8d88-daa01b90ee25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:56.512736 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad10f9-63f2-4370-8d88-daa01b90ee25 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:56.578598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 431/864] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:41:56 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:41:58.720000 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab6254-fa87-4557-a623-64d3769f1d5e None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:58.720577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab6254-fa87-4557-a623-64d3769f1d5e None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:58.720577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab6254-fa87-4557-a623-64d3769f1d5e None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:58.721877 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 434/865] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:41:58.727056 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:58.727494 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:58.727654 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:58.746070 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:41:58.747260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:41:58.800257 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:58.813785 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 432/866] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:41:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:41:59.078006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab6254-fa87-4557-a623-64d3769f1d5e None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:59.178013 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab6254-fa87-4557-a623-64d3769f1d5e None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:59.211171 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:59.211857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:59.212143 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:59.212421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:59.222930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:41:59.227872 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:41:59.229084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:41:59.230061 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 435/867] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:41:59.235435 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:59.331412 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6e672-c805-4e5e-900d-6ffa404de949 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:59.364757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:59.365355 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:59.365643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:59.365920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:59.366579 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:41:59.366888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:59.368287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:59.376198 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 433/868] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:59.383117 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:59.431818 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:59.480348 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c791-0e71-4070-8f45-fbd7f1cb1317 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:59.513086 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:59.513700 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:59.513985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:59.514257 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] PATH_INFO: `/v3/users/6678bdddcb7046dfa1db9ba45369101c` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:59.526697 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=6678bdddcb7046dfa1db9ba45369101c)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:59.528145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:59.529702 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 436/869] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:41:59 2022] GET /identity/v3/users/6678bdddcb7046dfa1db9ba45369101c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:59.535953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:59.585052 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:59.634066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d9c31-5fe2-4b16-beeb-4f743757ae1a None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:59.666431 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb97f8a-f892-4f38-ac69-6adc75e81382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:59.667049 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb97f8a-f892-4f38-ac69-6adc75e81382 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:59.667358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb97f8a-f892-4f38-ac69-6adc75e81382 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:59.667642 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb97f8a-f892-4f38-ac69-6adc75e81382 None admin] PATH_INFO: `/v3/projects/5665f18e26264425981084e038d9bd36` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:59.668757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb97f8a-f892-4f38-ac69-6adc75e81382 None admin] RBAC: Authorizing `identity:get_project(project_id=5665f18e26264425981084e038d9bd36)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:59.670177 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb97f8a-f892-4f38-ac69-6adc75e81382 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:59.672634 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 434/870] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:41:59 2022] GET /identity/v3/projects/5665f18e26264425981084e038d9bd36 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:41:59.678337 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:41:59.773547 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cea83-0736-4faf-a6ed-e631f831f3c7 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:41:59.781125 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:41:59.807575 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575eee-86c0-4096-bc62-41112fe6fe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:41:59.808226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575eee-86c0-4096-bc62-41112fe6fe88 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:41:59.808510 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575eee-86c0-4096-bc62-41112fe6fe88 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:41:59.808786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575eee-86c0-4096-bc62-41112fe6fe88 None admin] PATH_INFO: `/v3/projects/5665f18e26264425981084e038d9bd36/users/6678bdddcb7046dfa1db9ba45369101c/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:41:59.810680 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8575eee-86c0-4096-bc62-41112fe6fe88 None admin] RBAC: Authorizing `identity:create_grant(project_id=5665f18e26264425981084e038d9bd36, user_id=6678bdddcb7046dfa1db9ba45369101c, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:41:59.812298 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8575eee-86c0-4096-bc62-41112fe6fe88 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:41:59.874741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 437/871] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:41:59 2022] PUT /identity/v3/projects/5665f18e26264425981084e038d9bd36/users/6678bdddcb7046dfa1db9ba45369101c/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:42:01.984712 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057729f0-66a5-4039-a707-1aac028361c7 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:01.985356 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057729f0-66a5-4039-a707-1aac028361c7 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:01.985356 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057729f0-66a5-4039-a707-1aac028361c7 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:01.986609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 435/872] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:42:01.991784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:01.992183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:01.992349 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:02.010956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:42:02.012192 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:42:02.076945 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 438/873] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:42:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:42:02.351485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057729f0-66a5-4039-a707-1aac028361c7 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:02.451619 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057729f0-66a5-4039-a707-1aac028361c7 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:02.468334 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:02.485068 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:02.485696 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:02.485982 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:02.486260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:02.497004 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:42:02.501888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:42:02.503223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:42:02.504532 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 436/874] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:42:02.509107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:02.598935 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:02.604526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cfbf8-dcbd-4286-a819-3ab82921c2d9 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:02.637156 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:02.637751 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:02.638033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:02.638304 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:02.638960 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:42:02.639263 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:02.641974 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:02.648447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 439/875] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:02.654279 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:02.752050 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50b96b-9bdf-4125-bc4b-d8ffb258a0a0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:02.784893 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:02.785505 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:02.785790 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:02.786068 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] PATH_INFO: `/v3/users/6678bdddcb7046dfa1db9ba45369101c` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:02.787169 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] RBAC: Authorizing `identity:get_user(user_id=6678bdddcb7046dfa1db9ba45369101c)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:02.788690 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:02.790252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 437/876] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:42:02 2022] GET /identity/v3/users/6678bdddcb7046dfa1db9ba45369101c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:02.796093 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:02.893786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ad6ed-e228-4e38-92dc-77f2040bef12 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:02.926601 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:02.927202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:02.927516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:02.927797 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:02.954217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:02.992623 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:02.994071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:03.088968 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 440/877] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:42:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:42:03.095667 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:03.142631 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:03.194379 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fdc99-619d-48a9-afd9-772984d8fbae None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:03.227827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd235a92-7266-4ac4-8577-3e4d9a036659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:03.228454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235a92-7266-4ac4-8577-3e4d9a036659 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:03.228741 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235a92-7266-4ac4-8577-3e4d9a036659 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:03.229021 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235a92-7266-4ac4-8577-3e4d9a036659 None admin] PATH_INFO: `/v3/projects/5665f18e26264425981084e038d9bd36` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:03.230138 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd235a92-7266-4ac4-8577-3e4d9a036659 None admin] RBAC: Authorizing `identity:get_project(project_id=5665f18e26264425981084e038d9bd36)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:03.231599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd235a92-7266-4ac4-8577-3e4d9a036659 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:03.233221 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 438/878] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:42:03 2022] GET /identity/v3/projects/5665f18e26264425981084e038d9bd36 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:03.239517 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:03.295351 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:03.335029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33503eee-5eb0-4572-92f4-110fdd806201 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:03.367113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:03.367727 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:03.368008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:03.368283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:03.369004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=5665f18e26264425981084e038d9bd36, user.id=6678bdddcb7046dfa1db9ba45369101c) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:42:03.369313 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:03.370724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231b5e3-c21b-46fb-b73a-9d7c5b97205b None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:03.376837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 441/879] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:42:03 2022] GET /identity/v3/role_assignments?user.id=6678bdddcb7046dfa1db9ba45369101c&scope.project.id=5665f18e26264425981084e038d9bd36&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:05.507089 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4266c-c0a3-4fab-8095-cd027e909863 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:05.507706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4266c-c0a3-4fab-8095-cd027e909863 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:05.507706 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4266c-c0a3-4fab-8095-cd027e909863 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:05.509011 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 439/880] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:42:05.514308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad31a00-ff1b-44fb-b1d9-f164bf69cac5 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:05.515350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad31a00-ff1b-44fb-b1d9-f164bf69cac5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:05.515466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad31a00-ff1b-44fb-b1d9-f164bf69cac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:05.533289 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-5ad31a00-ff1b-44fb-b1d9-f164bf69cac5 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:42:05.534472 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-5ad31a00-ff1b-44fb-b1d9-f164bf69cac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:42:05.599500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 442/881] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:42:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:42:05.868034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4266c-c0a3-4fab-8095-cd027e909863 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:05.966958 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4266c-c0a3-4fab-8095-cd027e909863 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:05.980703 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:05.999806 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8937d7-cb99-4989-85c4-7900269ae088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:06.001055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8937d7-cb99-4989-85c4-7900269ae088 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:06.001055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8937d7-cb99-4989-85c4-7900269ae088 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:06.001055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8937d7-cb99-4989-85c4-7900269ae088 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:06.001637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8937d7-cb99-4989-85c4-7900269ae088 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:06.002988 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8937d7-cb99-4989-85c4-7900269ae088 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:06.065075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 440/882] 10.222.0.13 () {60 vars in 1138 bytes} [Mon Oct 17 05:42:05 2022] POST /identity/v3/domains => generated 267 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:42:08.201274 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44cf66-606d-44bd-8019-3dcae1d2aec5 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:08.201854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44cf66-606d-44bd-8019-3dcae1d2aec5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:08.201854 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44cf66-606d-44bd-8019-3dcae1d2aec5 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:08.203707 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 443/883] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:42:08.208986 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:08.209441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:08.209544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:08.228157 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:42:08.229352 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:42:08.293954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 441/884] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:42:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:42:08.560878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44cf66-606d-44bd-8019-3dcae1d2aec5 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:08.614006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:08.660795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44cf66-606d-44bd-8019-3dcae1d2aec5 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:08.694123 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40dd16-ed63-4274-8dc1-a428ec8d5707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:08.694750 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40dd16-ed63-4274-8dc1-a428ec8d5707 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:08.695040 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40dd16-ed63-4274-8dc1-a428ec8d5707 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:08.695345 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40dd16-ed63-4274-8dc1-a428ec8d5707 None admin] PATH_INFO: `/v3/domains/05fc3c0556b340c59edc91a623fd09b4` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:08.705497 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40dd16-ed63-4274-8dc1-a428ec8d5707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=05fc3c0556b340c59edc91a623fd09b4)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:08.706944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40dd16-ed63-4274-8dc1-a428ec8d5707 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:08.708545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 444/885] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Oct 17 05:42:08 2022] GET /identity/v3/domains/05fc3c0556b340c59edc91a623fd09b4 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:08.714289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:08.811170 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e585-f0ce-448f-a28d-457d5c593433 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:08.813483 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:08.845433 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419fba6-9e45-4136-a9fa-05f09dadb3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:08.846033 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419fba6-9e45-4136-a9fa-05f09dadb3f5 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:08.846322 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419fba6-9e45-4136-a9fa-05f09dadb3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:08.846604 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419fba6-9e45-4136-a9fa-05f09dadb3f5 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:08.847259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419fba6-9e45-4136-a9fa-05f09dadb3f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:08.848787 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419fba6-9e45-4136-a9fa-05f09dadb3f5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:08.932969 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 442/886] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:42:08 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:42:11.107659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398720b-7671-4f06-8934-94f81448d207 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:11.108693 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398720b-7671-4f06-8934-94f81448d207 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:11.108805 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398720b-7671-4f06-8934-94f81448d207 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:11.110094 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 445/887] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:42:11.115340 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:11.115724 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:11.115909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:11.134503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:42:11.135732 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:42:11.200685 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 443/888] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:42:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:42:11.468933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398720b-7671-4f06-8934-94f81448d207 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:11.568604 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398720b-7671-4f06-8934-94f81448d207 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:11.581307 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:11.601134 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f693d-2226-44aa-b2c3-3f5c1adf2606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:11.601752 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f693d-2226-44aa-b2c3-3f5c1adf2606 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:11.602039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f693d-2226-44aa-b2c3-3f5c1adf2606 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:11.602320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f693d-2226-44aa-b2c3-3f5c1adf2606 None admin] PATH_INFO: `/v3/domains/05fc3c0556b340c59edc91a623fd09b4` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:11.603478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f693d-2226-44aa-b2c3-3f5c1adf2606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=05fc3c0556b340c59edc91a623fd09b4)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:11.604909 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f693d-2226-44aa-b2c3-3f5c1adf2606 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:11.606481 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 446/889] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Oct 17 05:42:11 2022] GET /identity/v3/domains/05fc3c0556b340c59edc91a623fd09b4 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:11.612266 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:11.694215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:11.709086 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3559332-55ee-4ef0-89aa-4b6688baa0c8 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:11.743375 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643069d-8c2a-43f8-b5ba-f66f2a28ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:11.743947 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643069d-8c2a-43f8-b5ba-f66f2a28ee1f None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:11.744428 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643069d-8c2a-43f8-b5ba-f66f2a28ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:11.744766 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643069d-8c2a-43f8-b5ba-f66f2a28ee1f None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:11.745769 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643069d-8c2a-43f8-b5ba-f66f2a28ee1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:11.747209 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643069d-8c2a-43f8-b5ba-f66f2a28ee1f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:11.829701 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 444/890] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:42:11 2022] POST /identity/v3/users => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:42:13.978697 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6c963-6fc6-4dd8-96e1-6811259473c7 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:13.979236 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6c963-6fc6-4dd8-96e1-6811259473c7 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:13.979368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6c963-6fc6-4dd8-96e1-6811259473c7 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:13.980638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 447/891] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:42:13.986412 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:13.986869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:13.986984 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:14.005534 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:42:14.006716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:42:14.072175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 445/892] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:42:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:42:14.339441 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6c963-6fc6-4dd8-96e1-6811259473c7 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:14.441272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6c963-6fc6-4dd8-96e1-6811259473c7 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:14.474091 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:14.474710 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:14.474994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:14.475271 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:14.486135 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:42:14.490889 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:42:14.492185 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:42:14.493350 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 448/893] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:42:14.497985 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:14.565750 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:14.593762 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a4b2a-2085-4827-b20c-abfbd6203a0c None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:14.626934 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:14.627707 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:14.627994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:14.628273 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:14.628936 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:42:14.629246 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:14.630604 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:14.638386 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 446/894] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:14.644602 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:14.685166 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:14.741539 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652f5ed-3452-4cb5-925e-b5940ff90fd1 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:14.773676 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:14.774285 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:14.774564 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:14.774839 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] PATH_INFO: `/v3/users/3f63e61d1ea2480f8660f239bcd15bdd` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:14.787212 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] RBAC: Authorizing `identity:get_user(user_id=3f63e61d1ea2480f8660f239bcd15bdd)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:14.788719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:14.790296 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 449/895] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:42:14 2022] GET /identity/v3/users/3f63e61d1ea2480f8660f239bcd15bdd => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:14.796679 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:14.895874 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284c933-63c0-4d4f-8107-f8dde2a2971d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:14.928932 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb02ae-aae5-4402-9220-918152206453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:14.929556 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb02ae-aae5-4402-9220-918152206453 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:14.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb02ae-aae5-4402-9220-918152206453 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:14.930122 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb02ae-aae5-4402-9220-918152206453 None admin] PATH_INFO: `/v3/projects/e854c34d7a0043df93988e220e97e3b8` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:14.931235 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb02ae-aae5-4402-9220-918152206453 None admin] RBAC: Authorizing `identity:get_project(project_id=e854c34d7a0043df93988e220e97e3b8)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:14.932685 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb02ae-aae5-4402-9220-918152206453 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:14.934280 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 447/896] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:42:14 2022] GET /identity/v3/projects/e854c34d7a0043df93988e220e97e3b8 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:14.940202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:15.034895 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:15.038586 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641429f8-758b-4e63-a5d5-94738d48a57d None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:15.071600 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbc9d-f1f8-4bbd-9816-4cf12b879c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:15.072253 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbc9d-f1f8-4bbd-9816-4cf12b879c2a None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:15.072538 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbc9d-f1f8-4bbd-9816-4cf12b879c2a None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:15.072818 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbc9d-f1f8-4bbd-9816-4cf12b879c2a None admin] PATH_INFO: `/v3/projects/e854c34d7a0043df93988e220e97e3b8/users/3f63e61d1ea2480f8660f239bcd15bdd/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:15.074725 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcbc9d-f1f8-4bbd-9816-4cf12b879c2a None admin] RBAC: Authorizing `identity:create_grant(project_id=e854c34d7a0043df93988e220e97e3b8, user_id=3f63e61d1ea2480f8660f239bcd15bdd, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:15.076186 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcbc9d-f1f8-4bbd-9816-4cf12b879c2a None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:15.148949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 450/897] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:42:14 2022] PUT /identity/v3/projects/e854c34d7a0043df93988e220e97e3b8/users/3f63e61d1ea2480f8660f239bcd15bdd/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:42:17.258511 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383332-6446-4757-ae39-fa2e7af6ee36 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:17.259062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383332-6446-4757-ae39-fa2e7af6ee36 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:17.259062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2383332-6446-4757-ae39-fa2e7af6ee36 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:17.260455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 448/898] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:42:17.265630 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:17.266086 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:17.266195 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:17.284109 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:42:17.285296 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:42:17.350035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 451/899] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:42:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:42:17.614577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2383332-6446-4757-ae39-fa2e7af6ee36 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:17.617521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:17.714599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2383332-6446-4757-ae39-fa2e7af6ee36 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:17.746856 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:17.747499 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:17.747786 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:17.748062 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:17.758270 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:42:17.762990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:42:17.764820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:42:17.766003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 449/900] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:42:17.770682 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:17.855844 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:17.867549 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123836a-7b54-4b1f-99ce-a2244efdad36 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:17.900459 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:17.901057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:17.901339 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:17.901612 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:17.902272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:42:17.902581 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:17.903974 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:17.911835 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 452/901] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:17.917801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:17.978231 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:18.017287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f0b75-c3e8-4d5f-9ebf-49c18eb55a54 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:18.050211 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:18.050832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:18.051120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:18.051437 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] PATH_INFO: `/v3/users/3f63e61d1ea2480f8660f239bcd15bdd` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:18.052564 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] RBAC: Authorizing `identity:get_user(user_id=3f63e61d1ea2480f8660f239bcd15bdd)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:18.053987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:18.055564 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 450/902] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:42:17 2022] GET /identity/v3/users/3f63e61d1ea2480f8660f239bcd15bdd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:18.061789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:18.158793 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a483d-e043-41d7-bfa4-81ff39cbe5bc None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:18.191347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:18.191935 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:18.192217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:18.192491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:18.210240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:18.259163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:18.260624 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:18.355074 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 453/903] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:42:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:42:18.361039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:18.459982 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33f629-0ac0-427c-8b18-e2dda3bf5779 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:18.487283 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:18.492978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c076809-47d4-45cf-8976-4f991e7200e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:18.493599 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c076809-47d4-45cf-8976-4f991e7200e5 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:18.493887 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c076809-47d4-45cf-8976-4f991e7200e5 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:18.494169 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c076809-47d4-45cf-8976-4f991e7200e5 None admin] PATH_INFO: `/v3/projects/e854c34d7a0043df93988e220e97e3b8` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:18.495285 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c076809-47d4-45cf-8976-4f991e7200e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e854c34d7a0043df93988e220e97e3b8)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:18.496743 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c076809-47d4-45cf-8976-4f991e7200e5 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:18.498347 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 451/904] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:42:18 2022] GET /identity/v3/projects/e854c34d7a0043df93988e220e97e3b8 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:18.504472 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:42:18.553399 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:18.601818 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977105a4-c30f-4eef-bed4-3c3333751a55 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:42:18.634154 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:42:18.634743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:42:18.635022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:42:18.635320 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:42:18.636042 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=e854c34d7a0043df93988e220e97e3b8, user.id=3f63e61d1ea2480f8660f239bcd15bdd) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:42:18.636354 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:42:18.637765 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae86e5-1dec-421a-a52b-f6339a8e7e92 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:42:18.643949 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 454/905] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:42:18 2022] GET /identity/v3/role_assignments?user.id=3f63e61d1ea2480f8660f239bcd15bdd&scope.project.id=e854c34d7a0043df93988e220e97e3b8&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:42:28.515535 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:42:28.641420 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=43 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:32.495561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d01a9-6c86-4007-977e-ac7a9d71de44 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:32.496705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d01a9-6c86-4007-977e-ac7a9d71de44 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:32.496705 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d01a9-6c86-4007-977e-ac7a9d71de44 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:32.497954 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 452/906] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:32.503280 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:32.503776 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:32.503897 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:32.522873 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:32.524103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:32.589994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 455/907] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:32.863824 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d01a9-6c86-4007-977e-ac7a9d71de44 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:32.967186 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d01a9-6c86-4007-977e-ac7a9d71de44 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:33.000460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:33.001086 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:33.001370 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:33.001648 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:33.012560 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:33.013999 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:33.028354 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:33.029776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:33.031260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:33.031260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:33.031260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:33.031260 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 453/908] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:33.035754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:33.134200 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c1ef-1489-42d0-a4bf-28d2ef3bbdda None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:33.162818 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:33.169148 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:33.169754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:33.170078 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:33.170358 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:33.171002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:33.171321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:33.172699 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0336a-a67d-4567-b8a3-a20d8266d661 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:33.183970 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 456/909] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:33.189659 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:33.242554 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:33.289246 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc17855-ded5-4bd6-8f12-ff1286afa526 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:33.323413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157563d0-38c3-49e1-9d6e-7dd61b1c4fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:33.324029 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157563d0-38c3-49e1-9d6e-7dd61b1c4fe9 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:33.324318 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157563d0-38c3-49e1-9d6e-7dd61b1c4fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:33.324597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157563d0-38c3-49e1-9d6e-7dd61b1c4fe9 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:33.325239 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157563d0-38c3-49e1-9d6e-7dd61b1c4fe9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:33.326664 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157563d0-38c3-49e1-9d6e-7dd61b1c4fe9 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:33.415488 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 454/910] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:43:33 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:43:35.600998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ccfda-6b0d-4bf8-bb8d-ba600aca61c5 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:35.601638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ccfda-6b0d-4bf8-bb8d-ba600aca61c5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:35.601638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ccfda-6b0d-4bf8-bb8d-ba600aca61c5 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:35.602905 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 457/911] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:35.608708 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:35.609148 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:35.609258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:35.628117 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:35.629328 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:35.696422 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 455/912] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:35.965878 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ccfda-6b0d-4bf8-bb8d-ba600aca61c5 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:36.066996 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ccfda-6b0d-4bf8-bb8d-ba600aca61c5 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:36.100360 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:36.100983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:36.101268 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:36.101581 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:36.112784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:43:36.117571 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:36.118706 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:36.119795 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 458/913] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:43:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:36.125393 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:36.163721 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:36.221494 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33a350-0053-4041-9698-735e7838aab0 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:36.255069 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:36.260717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:36.260717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:36.260717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:36.260717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:36.260717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:36.260717 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:36.267013 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 456/914] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:36.272789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:36.322570 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:36.371930 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e93af-88a6-4540-88d3-9f5c61cd8741 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:36.404724 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:36.405389 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:36.405672 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:36.405950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:36.416632 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:36.418104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:36.432256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:36.433950 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:36.440406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:36.440406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:36.440406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:36.440406 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 459/915] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:36.440738 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:36.541680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:36.541680 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc9c2-1d01-4e94-b8dc-10401bbdbad6 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:36.575711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:36.576329 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:36.576618 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:36.576902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:36.577557 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:36.577878 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:36.579279 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:36.591817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 457/916] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:36.597628 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:36.699545 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcd64c-e0a4-43da-82fb-268f78e1a218 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:36.701048 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:36.733133 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:36.733755 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:36.734037 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:36.734314 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] PATH_INFO: `/v3/users/placement` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:36.741527 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:36.744573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:43:36.754634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:36.755963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:36.757096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 460/917] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:36.761869 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:36.865213 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f01feb-0e73-4aa0-b87a-e005e74f491b None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:36.898366 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:36.898976 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:36.899264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:36.899604 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:36.900344 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:36.900641 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:36.902591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:36.919469 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 458/918] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:43:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:36.925589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:37.024386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c0b31-fb10-4c8a-bb3e-44f1ad564575 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:37.058983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:37.059592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:37.059905 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:37.060187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:37.070544 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:37.072021 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:37.086004 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:37.087879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:37.088926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:37.088926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:37.088926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:37.088926 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 461/919] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:37.093553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:37.112896 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:37.192116 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69069d94-8a4c-43ea-a0bb-f274e03ab577 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:37.226749 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:37.227404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:37.227698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:37.227992 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:37.228643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:37.229005 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:37.230432 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:37.241700 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 459/920] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:37.247514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:37.264190 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:37.347369 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af352d-8988-4dbe-bb8c-2de99fa97678 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:37.380841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:37.381456 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:37.381880 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:37.382103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:37.388742 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:37.390179 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:43:37.401961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:37.411658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:37.412705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 462/921] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:37.413313 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:37.488516 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:37.509810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46169bb8-61f2-43bf-aaca-767e420c4d64 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:37.543169 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:37.543801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:37.544088 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:37.544368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:37.545052 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:37.545366 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:37.546792 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f7bb5-685b-4d01-b582-2ad43d542b23 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:37.559394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 460/922] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:37.565466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:37.640981 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:37.665907 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8e57-7a1d-49af-b372-97e05a11e7aa None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:37.699467 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dea05e-4832-421a-af81-abb387066e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:37.700115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dea05e-4832-421a-af81-abb387066e38 None admin] REQUEST_METHOD: `PUT` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:37.700701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dea05e-4832-421a-af81-abb387066e38 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:37.700701 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dea05e-4832-421a-af81-abb387066e38 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/c44be36d3a094a048ee37701ff21ed32/roles/345eb0b49bc54832909c28b8be23267e` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:37.715403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dea05e-4832-421a-af81-abb387066e38 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=c44be36d3a094a048ee37701ff21ed32, role_id=345eb0b49bc54832909c28b8be23267e)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:37.716851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dea05e-4832-421a-af81-abb387066e38 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:37.930783 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 463/923] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:43:37 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/c44be36d3a094a048ee37701ff21ed32/roles/345eb0b49bc54832909c28b8be23267e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:43:40.068496 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f12da0-c8bd-4317-9113-e8d554ae7af2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:40.069038 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f12da0-c8bd-4317-9113-e8d554ae7af2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:40.069038 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f12da0-c8bd-4317-9113-e8d554ae7af2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:40.070314 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 461/924] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:40.076276 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:40.076717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:40.076826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:40.094638 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:40.095875 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:40.162369 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 464/925] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:40.433979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f12da0-c8bd-4317-9113-e8d554ae7af2 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:40.538088 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f12da0-c8bd-4317-9113-e8d554ae7af2 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:40.574143 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:40.574780 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:40.575066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:40.575377 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:40.582343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:40.586276 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:43:40.591284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:40.592544 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:40.593683 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 462/926] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 05:43:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:40.598568 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:40.653851 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:40.699414 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5ede6-a845-4365-97e4-7f7d4e33243d None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:40.733267 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:40.733876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:40.734160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:40.734438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:40.735104 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] RBAC: Adding query filter params (name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:40.735451 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:40.736822 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:40.744741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 465/927] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 05:43:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:40.751120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:40.853648 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29834299-bd3a-4b39-b54b-91d7de7f04b0 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:40.889174 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:40.889810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:40.890100 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:40.890380 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:40.900387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:40.902382 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:40.916666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:40.923620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:40.925215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:40.925215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:40.925215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:40.925215 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 463/928] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:40.925540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:41.022289 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b36b2-ea8a-4055-97d5-6eabd746cad3 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:41.038399 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:41.056057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:41.056665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:41.056947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:41.057222 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:41.057913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:41.058219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:41.059615 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:41.071629 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 466/929] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:41.077742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:41.131617 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:41.182875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34972c-fcdd-4690-8440-61b8e03be858 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:41.216699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:41.217331 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:41.217620 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:41.217904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:41.225281 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:41.226735 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:43:41.237864 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:41.239051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:41.240254 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 464/930] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:41.244881 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:41.343593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f7486-62b1-482b-b806-e09bbab80429 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:41.377933 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:41.378539 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:41.378820 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:41.379096 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:41.379929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:41.380240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:41.381660 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:41.398894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 467/931] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:43:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:41.405502 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:41.508402 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d825c-0795-475f-ab73-3b3a4c2571e6 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:41.519112 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:41.542253 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:41.543479 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:41.543953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:41.544403 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:41.610368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:41.611778 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:41.708444 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 465/932] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:43:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:43:41.714786 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:41.748327 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:41.812505 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80018b-19e1-4e9b-b31c-2f005f297a2c None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:41.846281 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:41.846909 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:41.847192 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:41.847491 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:41.857794 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:41.859252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:41.873986 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:41.875064 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:41.876155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:41.876155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:41.876155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:41.876155 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 468/933] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:41.880782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:41.892250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:41.978804 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316d287-9b8d-4279-8720-669749b1614e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:42.013567 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:42.014179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:42.014467 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:42.014750 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:42.015418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:42.015730 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:42.017109 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:42.029003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 466/934] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:42.034863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:42.126473 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:42.142033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac70f6-b385-41e2-8262-fa91cd314115 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:42.181931 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:42.182566 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:42.182890 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:42.183174 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:42.190218 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:42.191687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:43:42.202985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:42.204082 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:42.205142 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 469/935] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:42.210120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:42.286039 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:42.319504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65667b46-4ef5-4e8e-9457-3ebc4ff63564 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:42.355606 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:42.356327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:42.356693 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:42.357042 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] PATH_INFO: `/v3/projects` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:42.357752 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:42.358066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:42.359576 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f084c-803f-4661-bf97-462cffc706b3 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:42.373050 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 467/936] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:42.383005 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:42.487187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d0c2f-b9a5-4d2a-883d-5a8f3066ef66 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:42.521084 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:42.521683 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:42.521966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:42.522240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:42.522967 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] RBAC: Adding query filter params (role.id=345eb0b49bc54832909c28b8be23267e, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=c44be36d3a094a048ee37701ff21ed32) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:42.523277 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:42.524719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fc616-60ef-4b0c-9de9-c62499a2b32c None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:42.530516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 470/937] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:43:42 2022] GET /identity/v3/role_assignments?user.id=c44be36d3a094a048ee37701ff21ed32&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=345eb0b49bc54832909c28b8be23267e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:44.698241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da911b-ae99-4c88-9c24-0d2b1c5cd1a2 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:44.698894 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da911b-ae99-4c88-9c24-0d2b1c5cd1a2 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:44.698894 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da911b-ae99-4c88-9c24-0d2b1c5cd1a2 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:44.700259 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 468/938] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:44.705613 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640726ba-4b89-4c56-8425-4050cba821de None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:44.706063 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640726ba-4b89-4c56-8425-4050cba821de None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:44.706165 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640726ba-4b89-4c56-8425-4050cba821de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:44.724826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-640726ba-4b89-4c56-8425-4050cba821de None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:44.726113 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-640726ba-4b89-4c56-8425-4050cba821de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:44.728973 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:44.793589 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 471/939] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:45.065517 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da911b-ae99-4c88-9c24-0d2b1c5cd1a2 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:45.165185 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da911b-ae99-4c88-9c24-0d2b1c5cd1a2 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:45.194754 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:45.200493 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:45.206909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:45.206909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:45.206909 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:45.211610 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:43:45.216902 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:45.218159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:45.219413 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 469/940] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:45.224033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640726ba-4b89-4c56-8425-4050cba821de None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:45.322577 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640726ba-4b89-4c56-8425-4050cba821de None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:45.357207 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:45.357805 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:45.358092 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:45.358368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] PATH_INFO: `/v3/roles` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:45.359035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:45.359370 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:45.360737 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:45.368618 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 472/941] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:45.374590 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:45.474817 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062cf78-58df-4ffd-bf6d-f89cac3fcfa8 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:45.508832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:45.509462 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:45.509752 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:45.510034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:45.520530 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:45.521971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:45.536935 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:45.538204 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:45.539481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:45.539481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:45.539481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:45.539481 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 470/942] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:45.543887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:45.546764 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:45.641616 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ee38b-0cd2-44f3-8c85-fa6ca1414b84 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:45.675231 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:45.675866 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:45.676153 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:45.676434 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:45.677087 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:45.677402 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:45.678831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:45.690528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 473/943] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:45.696452 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:45.741908 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:45.798269 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c56ef4-8a8f-4dc3-87b9-bfdd7873851f None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:45.833500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:45.834130 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:45.834416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:45.834695 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:45.843358 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:45.844834 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:43:45.855924 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:45.857144 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:45.858394 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 471/944] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:45.863209 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:45.925297 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:45.962855 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d412ac4-187d-44d8-8704-f05474e8b5d5 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:45.997596 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:45.998198 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:45.998526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:45.998799 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] PATH_INFO: `/v3/users` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:45.999550 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:45.999859 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:46.001346 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:46.018743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 474/945] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:43:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:46.024903 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:46.124327 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10a863-5051-49bb-87bc-86a3bd620d4d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:46.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:46.158206 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:46.158918 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:46.159202 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:46.159507 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:46.170264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:46.171725 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:46.186365 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:46.187691 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:46.188966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:46.188966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:46.188966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:46.188966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 472/946] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:46.193341 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:46.294100 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f46c-d5b0-4b36-9b34-295640b03932 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:46.314593 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:46.328753 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:46.329357 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:46.329642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:46.329925 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:46.330565 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:46.330876 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:46.332272 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:46.343892 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 475/947] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:46.349755 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:46.448252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e8daa-e64a-42b4-bbe5-d369fe6fc077 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:46.483422 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:46.484051 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:46.484387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:46.484665 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:46.491229 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:46.498091 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:43:46.503794 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:46.504978 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:46.506055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 473/948] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:46.511528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:46.609422 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d89d2-0aa3-4a69-8d26-c7c94a0510ea None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:46.642845 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:46.643461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:46.643752 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:46.644035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:46.644727 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:46.645039 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:46.646465 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34a8bf-26e2-4293-92f9-bd885a0c5ccb None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:46.658807 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 476/949] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:46.664948 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:46.681175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:46.765675 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a227-b4ef-4d43-be50-023358c85fbc None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:46.799325 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004f372-b8ba-4d95-b08c-e6d10eb92ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:46.800025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004f372-b8ba-4d95-b08c-e6d10eb92ca4 None admin] REQUEST_METHOD: `PUT` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:46.800310 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004f372-b8ba-4d95-b08c-e6d10eb92ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:46.800588 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004f372-b8ba-4d95-b08c-e6d10eb92ca4 None admin] PATH_INFO: `/v3/projects/6e1d626bfda44be3804dced335b709b8/users/c44be36d3a094a048ee37701ff21ed32/roles/df4da7cff6734799a09e0b770b60845f` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:46.802492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004f372-b8ba-4d95-b08c-e6d10eb92ca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e1d626bfda44be3804dced335b709b8, user_id=c44be36d3a094a048ee37701ff21ed32, role_id=df4da7cff6734799a09e0b770b60845f)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:46.803963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004f372-b8ba-4d95-b08c-e6d10eb92ca4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:46.913402 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 474/950] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:43:46 2022] PUT /identity/v3/projects/6e1d626bfda44be3804dced335b709b8/users/c44be36d3a094a048ee37701ff21ed32/roles/df4da7cff6734799a09e0b770b60845f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:43:49.062438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10b86-9a56-49e1-b18d-81ea49bd642e None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:49.062983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10b86-9a56-49e1-b18d-81ea49bd642e None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:49.062983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10b86-9a56-49e1-b18d-81ea49bd642e None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:49.064401 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 477/951] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:49.069886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:49.070464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:49.070464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:49.088504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:49.090239 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:49.155757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 475/952] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:49.429146 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10b86-9a56-49e1-b18d-81ea49bd642e None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:49.471329 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:49.528591 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10b86-9a56-49e1-b18d-81ea49bd642e None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:49.562130 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:49.562752 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:49.563033 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:49.563333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:49.573748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:43:49.578502 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:49.583219 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:49.584421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 478/953] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:43:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:49.585783 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:49.589001 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:49.684688 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a4292-241e-4081-9830-30020ca1a107 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:49.719643 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:49.723376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:49.723376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:49.723376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] PATH_INFO: `/v3/roles` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:49.723376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] RBAC: Adding query filter params (name=admin) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:49.723376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:49.723376 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:49.731710 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 476/954] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:43:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:49.737723 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:49.835837 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:49.838765 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d72327-ae3b-4f75-95ef-621a5ce8a063 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:49.872368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:49.872983 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:49.873264 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:49.873539 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:49.883887 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:49.885317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:49.899022 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:49.900283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:49.902120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:49.902120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:49.902120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:49.903447 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 479/955] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:49.906561 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:49.972677 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:50.007074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281561-b084-461c-b97d-d7fd8229c59e None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:50.040061 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:50.040681 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:50.040971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:50.041252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] PATH_INFO: `/v3/domains` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:50.042416 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:50.042734 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:50.044199 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:50.056149 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 477/956] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:50.061992 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:50.161918 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd3b80-e147-4f76-b8fa-524083bcaea8 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:50.195864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:50.196483 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:50.196766 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:50.197041 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] PATH_INFO: `/v3/users/placement` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:50.204423 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:50.205847 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 05:43:50.216789 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:50.218029 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:50.219438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 480/957] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:50.223966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:50.322060 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e957f-116f-4731-bc24-693a987e2195 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:50.347460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:50.356751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:50.357362 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:50.357647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:50.357969 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] PATH_INFO: `/v3/users` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:50.358814 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:50.359129 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:50.360591 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:50.377990 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 478/958] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 05:43:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:50.383749 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:50.398994 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:50.483516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca96de-e451-467f-9521-f69a9538e234 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:50.517174 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:50.517775 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:50.518057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:50.518333 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:50.584709 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:50.586151 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:50.600120 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:50.685650 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 481/959] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 05:43:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 05:43:50.691672 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:50.790656 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a92c-3db5-41a6-8165-b5449319cb2d None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:50.826449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:50.827075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:50.827387 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:50.827667 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:50.837546 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:50.839659 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:43:50.853972 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:50.855261 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:43:50.856363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:43:50.856363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:43:50.856363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:50.856363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 479/960] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:50.861056 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:50.958998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67a1c3-fd43-460a-b76d-cd0b66414c75 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:50.971793 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:50.992375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:50.992966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:50.993249 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:50.993525 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] PATH_INFO: `/v3/domains` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:50.994173 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:50.994480 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:50.996417 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:51.007377 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 482/961] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:51.013105 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:51.048832 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:51.112472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d85a6-4b6f-4c12-b381-eb23f8134c98 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:51.146714 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:51.147469 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:51.147754 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:51.148029 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:51.154961 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:51.156426 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 05:43:51.168336 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:51.171223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: ERROR keystone.server.flask.application Oct 17 05:43:51.172320 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 480/962] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:51.176578 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:51.276386 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b192b-813f-4236-9410-085c619c4482 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:51.309648 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:51.310183 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:51.310464 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:51.310747 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] PATH_INFO: `/v3/projects` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:51.311467 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:51.311777 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:51.313317 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d340f-d46a-4c19-835b-a711151eceec None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:51.325649 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 483/963] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 05:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:51.331896 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:51.425455 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:51.431183 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6e5e5-c1ea-4640-8241-a21a702495f4 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:51.465372 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:51.466110 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:51.466391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:51.466741 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:51.467537 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] RBAC: Adding query filter params (role.id=df4da7cff6734799a09e0b770b60845f, scope.project.id=6e1d626bfda44be3804dced335b709b8, user.id=c44be36d3a094a048ee37701ff21ed32) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:51.467851 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:51.469265 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf3ae3-3ce2-472d-b56d-819b38f00c76 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:51.474938 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 481/964] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 05:43:51 2022] GET /identity/v3/role_assignments?user.id=c44be36d3a094a048ee37701ff21ed32&scope.project.id=6e1d626bfda44be3804dced335b709b8&role.id=df4da7cff6734799a09e0b770b60845f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:53.639675 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7f61b-660c-44fd-b7cf-6b53ad259fc1 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:53.640229 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7f61b-660c-44fd-b7cf-6b53ad259fc1 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:53.640330 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7f61b-660c-44fd-b7cf-6b53ad259fc1 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:53.641595 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 484/965] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:53.647074 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:53.647521 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:53.647687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:53.666289 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:53.668148 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:53.734232 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 482/966] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:54.004931 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7f61b-660c-44fd-b7cf-6b53ad259fc1 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:54.036923 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:54.106905 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7f61b-660c-44fd-b7cf-6b53ad259fc1 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:54.140956 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:54.141578 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:54.141861 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:54.142138 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] PATH_INFO: `/v3/services/placement` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:54.142717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:54.144112 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:54.153300 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:54.154500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:54.155513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 485/967] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:43:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:54.160460 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:54.249023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:54.258492 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cff70-1e91-4591-9c8d-a89cba6f112c None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:54.293263 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:54.293870 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:54.294159 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:54.294438 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:54.295133 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] RBAC: Adding query filter params (name=placement) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:54.295495 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:54.296867 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2139d2-8666-4ab6-b909-2ff8e3d65e1e None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:54.302072 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 483/968] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:43:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:43:54.308466 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:54.403514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:54.409259 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ed24a-73ab-4d37-8261-4d2e52bc3d6e None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:54.443659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:54.444321 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:54.444612 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:54.444895 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] PATH_INFO: `/v3/services` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:54.445528 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] RBAC: Adding query filter params (type=placement) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:54.445841 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:54.447226 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a8021-7724-4199-a118-81f60efc90fd None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:54.452214 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 486/969] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:43:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:56.606152 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4d63b-249d-431a-bbca-3c35b712acdd None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:56.606716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4d63b-249d-431a-bbca-3c35b712acdd None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:56.606716 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4d63b-249d-431a-bbca-3c35b712acdd None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:56.608146 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 484/970] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:56.613457 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166981d1-6f71-4629-a446-0de86bc1c847 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:56.613852 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166981d1-6f71-4629-a446-0de86bc1c847 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:56.614057 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166981d1-6f71-4629-a446-0de86bc1c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:56.632650 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-166981d1-6f71-4629-a446-0de86bc1c847 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:56.633868 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-166981d1-6f71-4629-a446-0de86bc1c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:56.701944 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 487/971] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:56.980454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4d63b-249d-431a-bbca-3c35b712acdd None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:57.083343 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4d63b-249d-431a-bbca-3c35b712acdd None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:57.116953 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11fedd-9c20-4de3-b642-c56394aeb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:57.117549 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11fedd-9c20-4de3-b642-c56394aeb9e6 None admin] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:57.118328 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11fedd-9c20-4de3-b642-c56394aeb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:57.118328 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11fedd-9c20-4de3-b642-c56394aeb9e6 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:57.118730 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11fedd-9c20-4de3-b642-c56394aeb9e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:57.120628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11fedd-9c20-4de3-b642-c56394aeb9e6 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:57.188065 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 485/972] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:43:56 2022] POST /identity/v3/services => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:43:59.317864 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af41e5d-ab3d-4546-9129-14086120471b None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:59.318526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af41e5d-ab3d-4546-9129-14086120471b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:59.318526 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af41e5d-ab3d-4546-9129-14086120471b None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:59.319809 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 488/973] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:43:59.325454 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:59.325898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:59.326006 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:59.344700 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:43:59.345970 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:43:59.355732 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:59.419994 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 486/974] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:43:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:43:59.690308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af41e5d-ab3d-4546-9129-14086120471b None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:59.781248 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:43:59.791631 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af41e5d-ab3d-4546-9129-14086120471b None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:59.825074 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:59.825696 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:59.825979 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:59.826254 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] PATH_INFO: `/v3/services/placement` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:59.826843 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:59.828240 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:43:59.836687 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:43:59.838301 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:43:59.839603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 489/975] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:43:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:43:59.843944 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:43:59.945252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea02da7-8832-45a4-8587-060a43b55112 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:43:59.979785 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:43:59.980391 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:43:59.980682 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:43:59.980963 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:43:59.981594 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] RBAC: Adding query filter params (name=placement) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:43:59.981904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:43:59.983280 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:43:59.988386 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 487/976] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:43:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:43:59.994500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:44:00.094167 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119012-07c2-45b9-9de9-31570f3372ec None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:44:00.128224 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:44:00.128829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:00.129115 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:00.129396 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:00.130103 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] RBAC: Adding query filter params (interface=public, service_id=034cf8bfc8844975922036484feda7e7, region_id=RegionOne) {{(pid=153118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:44:00.130418 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:44:00.131848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729efd20-4d67-4b4f-a769-7c736ad8f2e2 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:44:00.136937 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 490/977] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 05:43:59 2022] GET /identity/v3/endpoints?service_id=034cf8bfc8844975922036484feda7e7&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:44:00.142760 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:44:00.146595 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:44:00.243371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065377a4-85af-4088-85d1-eb3abb2c9254 None admin] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:44:00.277264 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dac18-6720-4260-a0c0-7bbc48e0727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:44:00.277873 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dac18-6720-4260-a0c0-7bbc48e0727b None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:00.278163 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dac18-6720-4260-a0c0-7bbc48e0727b None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:00.278445 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dac18-6720-4260-a0c0-7bbc48e0727b None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:00.279053 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dac18-6720-4260-a0c0-7bbc48e0727b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:44:00.280468 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dac18-6720-4260-a0c0-7bbc48e0727b None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:44:00.287290 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 488/978] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 05:44:00 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 05:44:02.419457 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea103-14e2-48e7-978e-47ee829a2b4e None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:02.420032 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea103-14e2-48e7-978e-47ee829a2b4e None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:02.420032 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ea103-14e2-48e7-978e-47ee829a2b4e None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:02.421425 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 491/979] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:44:02.426856 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:02.427250 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:02.427449 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:02.445395 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:44:02.446589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:44:02.508314 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:44:02.514351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 489/980] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:44:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 05:44:02.784620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea103-14e2-48e7-978e-47ee829a2b4e None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:44:02.804904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:44:02.886965 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ea103-14e2-48e7-978e-47ee829a2b4e None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:44:02.920038 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:44:02.920647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:02.920931 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:02.921216 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] PATH_INFO: `/v3/services/placement` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:02.921804 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:44:02.923187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:44:02.933071 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:44:02.934917 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application with Lock( Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: ERROR keystone.server.flask.application Oct 17 05:44:02.936665 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 492/981] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 05:44:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:44:02.940384 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:44:03.038344 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7ed5-d2b9-438c-baf6-5055b957ac93 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:44:03.072553 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:44:03.073692 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:03.073980 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:03.074258 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] PATH_INFO: `/v3/services` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:03.074886 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] RBAC: Adding query filter params (name=placement) {{(pid=153119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:44:03.075196 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:44:03.076609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f4cb1-4d66-4f54-a679-bf309317124d None admin] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:44:03.081658 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 490/982] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 05:44:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:44:03.087603 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:44:03.170873 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:44:03.190214 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e046d0-adce-441e-84e9-133a7611a277 None admin] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:44:03.224233 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731d88b-a556-4eb0-b98e-26729f57ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '437c2d620e4648bb9b558b4098744fa0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:44:03.224882 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731d88b-a556-4eb0-b98e-26729f57ce83 None admin] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:44:03.225172 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731d88b-a556-4eb0-b98e-26729f57ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:44:03.225454 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731d88b-a556-4eb0-b98e-26729f57ce83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:44:03.226163 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731d88b-a556-4eb0-b98e-26729f57ce83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:44:03.227669 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731d88b-a556-4eb0-b98e-26729f57ce83 None admin] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:44:03.297896 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 493/983] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 05:44:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:44:13.078707 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=69 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:44:13.293145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:28.780023 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2ab79-2e20-400c-a305-1d751ffe5526 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:28.780577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2ab79-2e20-400c-a305-1d751ffe5526 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:28.780577 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2ab79-2e20-400c-a305-1d751ffe5526 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:28.787161 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02a4b4-1eed-4466-ae0b-fdb2f81a1c64 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:28.787644 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02a4b4-1eed-4466-ae0b-fdb2f81a1c64 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:28.787850 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02a4b4-1eed-4466-ae0b-fdb2f81a1c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:28.793112 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 491/984] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:28 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:28.902403 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-0d02a4b4-1eed-4466-ae0b-fdb2f81a1c64 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:28.904298 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-0d02a4b4-1eed-4466-ae0b-fdb2f81a1c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:29.135587 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 494/985] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:52:29.451212 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2458-dd8b-4927-bc9c-d51001e88fd1 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:29.451584 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2458-dd8b-4927-bc9c-d51001e88fd1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:29.451875 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2458-dd8b-4927-bc9c-d51001e88fd1 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:29.453237 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 492/986] 10.222.0.13 () {54 vars in 964 bytes} [Mon Oct 17 05:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:29.460421 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d08bf5-97d7-48ea-aa90-c45547f37d7b None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:29.460708 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d08bf5-97d7-48ea-aa90-c45547f37d7b None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:29.460998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d08bf5-97d7-48ea-aa90-c45547f37d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:29.632573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-b1d08bf5-97d7-48ea-aa90-c45547f37d7b None None] MFA Rules not processed for user `6f280acb87494dc59f0f7c222edcc87e`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:29.633829 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-b1d08bf5-97d7-48ea-aa90-c45547f37d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:29.674894 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:29.771816 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 495/987] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Oct 17 05:52:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:29.783971 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2458-dd8b-4927-bc9c-d51001e88fd1 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:29.786663 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-73cb2458-dd8b-4927-bc9c-d51001e88fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:29.955810 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2458-dd8b-4927-bc9c-d51001e88fd1 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:30.055175 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:30.056404 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:30.057575 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:30.058066 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:30.059888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:30.126558 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:30.194742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:30.196751 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75d903-992c-4564-a559-671c9517ab86 service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:30.232059 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 493/988] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:32.734495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb049a-1655-4e0f-af1a-ebb19df57050 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:32.735059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb049a-1655-4e0f-af1a-ebb19df57050 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:32.735059 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb049a-1655-4e0f-af1a-ebb19df57050 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:32.736450 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 496/989] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:32.741615 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f84b4-9973-4b0f-ac9d-b0a28119a0a0 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:32.742031 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f84b4-9973-4b0f-ac9d-b0a28119a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:32.742176 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f84b4-9973-4b0f-ac9d-b0a28119a0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:32.826060 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-125f84b4-9973-4b0f-ac9d-b0a28119a0a0 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:32.827220 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-125f84b4-9973-4b0f-ac9d-b0a28119a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:32.923166 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 494/990] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:52:33.204558 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb049a-1655-4e0f-af1a-ebb19df57050 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:33.206177 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-20cb049a-1655-4e0f-af1a-ebb19df57050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:33.309495 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb049a-1655-4e0f-af1a-ebb19df57050 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:33.380534 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:33.383026 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:33.383658 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:33.383940 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:33.384217 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:33.385823 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:33.496998 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:33.498368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aa25f-5a91-4425-83cf-c633ed2eb62d service neutron] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:33.516178 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 497/991] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:36.390162 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2059-4d01-4927-a2bf-546bb0002d76 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:36.390709 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2059-4d01-4927-a2bf-546bb0002d76 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:36.390709 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d2059-4d01-4927-a2bf-546bb0002d76 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:36.392094 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 495/992] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:36.397308 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cadc8-ddc6-463c-880a-e58ff439c07a None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:36.397720 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cadc8-ddc6-463c-880a-e58ff439c07a None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:36.397900 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cadc8-ddc6-463c-880a-e58ff439c07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:36.464250 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-f16cadc8-ddc6-463c-880a-e58ff439c07a None None] MFA Rules not processed for user `dc60bb94695043b38502ba313cbe55ad`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:36.465540 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-f16cadc8-ddc6-463c-880a-e58ff439c07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:36.516362 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:36.561072 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 498/993] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:36.845153 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2059-4d01-4927-a2bf-546bb0002d76 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:36.846697 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-601d2059-4d01-4927-a2bf-546bb0002d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:36.857399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:36.950892 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d2059-4d01-4927-a2bf-546bb0002d76 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:37.025797 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:37.026400 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:37.026678 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:37.026950 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:37.028647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:37.089585 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:37.135118 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:37.136573 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0844c-b1b7-4138-a20b-78fc7a191b50 service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:37.155135 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 496/994] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:40.115058 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd9499-e7e3-43e8-9296-07235b47d234 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:40.115620 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd9499-e7e3-43e8-9296-07235b47d234 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:40.115719 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd9499-e7e3-43e8-9296-07235b47d234 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:40.117055 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 499/995] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:40.122257 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d44a-fc64-42b7-936a-e4a0640fbcde None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:40.122670 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d44a-fc64-42b7-936a-e4a0640fbcde None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:40.122819 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84d44a-fc64-42b7-936a-e4a0640fbcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:40.187901 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-cf84d44a-fc64-42b7-936a-e4a0640fbcde None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:40.189101 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-cf84d44a-fc64-42b7-936a-e4a0640fbcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:40.284044 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 497/996] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:40.561610 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd9499-e7e3-43e8-9296-07235b47d234 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:40.563154 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-e6dd9499-e7e3-43e8-9296-07235b47d234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:40.664934 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd9499-e7e3-43e8-9296-07235b47d234 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:40.739038 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:40.739743 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:40.740031 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:40.740311 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:40.741914 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:40.749387 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:40.852500 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:40.853977 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708d65b-42d7-4798-a048-6f9d1dbfbb2d service neutron] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:40.871844 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 500/997] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:44.029667 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39a3d-385d-4246-bc46-a1604533665f None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:44.030210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39a3d-385d-4246-bc46-a1604533665f None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:44.030210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39a3d-385d-4246-bc46-a1604533665f None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:44.031666 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 498/998] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:44.037392 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d81d4-6a8c-4874-8b2e-7ca4a1205fa5 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:44.037791 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d81d4-6a8c-4874-8b2e-7ca4a1205fa5 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:44.037947 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d81d4-6a8c-4874-8b2e-7ca4a1205fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:44.107569 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-040d81d4-6a8c-4874-8b2e-7ca4a1205fa5 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:44.108826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-040d81d4-6a8c-4874-8b2e-7ca4a1205fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:44.172642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:44.213036 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 501/999] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:44.511055 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04185fa-036e-4a99-818e-a6dbdb8d81c5 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:44.511271 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04185fa-036e-4a99-818e-a6dbdb8d81c5 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:44.511589 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04185fa-036e-4a99-818e-a6dbdb8d81c5 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:44.512920 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 499/1000] 10.222.0.13 () {54 vars in 964 bytes} [Mon Oct 17 05:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:44.518919 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea14987-be06-4ce0-bb83-ca2d4f880769 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:44.519283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea14987-be06-4ce0-bb83-ca2d4f880769 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:44.519659 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea14987-be06-4ce0-bb83-ca2d4f880769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:44.597426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-fea14987-be06-4ce0-bb83-ca2d4f880769 None None] MFA Rules not processed for user `6f280acb87494dc59f0f7c222edcc87e`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:44.598647 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-fea14987-be06-4ce0-bb83-ca2d4f880769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:44.674283 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 502/1001] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Oct 17 05:52:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:44.683628 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04185fa-036e-4a99-818e-a6dbdb8d81c5 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:44.685154 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-d04185fa-036e-4a99-818e-a6dbdb8d81c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:44.705257 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:44.792351 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04185fa-036e-4a99-818e-a6dbdb8d81c5 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:44.867103 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:44.867742 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:44.868030 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:44.868308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:44.869923 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:44.946371 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:44.986474 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:44.988071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6eb61-fa5f-4f9f-836a-96bbaf6368fa service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:45.006087 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 500/1002] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:47.451284 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd0778-ccd8-49e3-9ad9-c557419bf497 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:47.451831 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd0778-ccd8-49e3-9ad9-c557419bf497 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:47.451939 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd0778-ccd8-49e3-9ad9-c557419bf497 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:47.453251 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 503/1003] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:47.458609 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b1a30-3262-4bea-9b10-1d97f1012bdb None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:47.459010 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b1a30-3262-4bea-9b10-1d97f1012bdb None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:47.459179 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b1a30-3262-4bea-9b10-1d97f1012bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:47.525466 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-d93b1a30-3262-4bea-9b10-1d97f1012bdb None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:47.526699 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-d93b1a30-3262-4bea-9b10-1d97f1012bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:47.623441 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 501/1004] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:47.902318 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd0778-ccd8-49e3-9ad9-c557419bf497 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:47.903899 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-58dd0778-ccd8-49e3-9ad9-c557419bf497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:47.965006 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:48.009353 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd0778-ccd8-49e3-9ad9-c557419bf497 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:48.082075 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:48.082611 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:48.082892 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:48.083168 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:48.084798 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:48.189879 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:48.192107 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:48.193548 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db37723-5b74-4ad7-9be9-114ab686b766 service neutron] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:48.211536 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 504/1005] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:50.376373 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e9383-9827-443f-a70f-bf7250cfcc9d None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:50.376904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e9383-9827-443f-a70f-bf7250cfcc9d None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:50.376904 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e9383-9827-443f-a70f-bf7250cfcc9d None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:50.378252 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 502/1006] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:50.383654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f23fd-2a7f-4138-b6bc-1a670a78eecd None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:50.384636 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f23fd-2a7f-4138-b6bc-1a670a78eecd None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:50.384807 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f23fd-2a7f-4138-b6bc-1a670a78eecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:50.451050 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-cc0f23fd-2a7f-4138-b6bc-1a670a78eecd None None] MFA Rules not processed for user `dc60bb94695043b38502ba313cbe55ad`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:50.452303 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-cc0f23fd-2a7f-4138-b6bc-1a670a78eecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:50.547546 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 505/1007] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:50.836880 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e9383-9827-443f-a70f-bf7250cfcc9d None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:50.838916 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-dd0e9383-9827-443f-a70f-bf7250cfcc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:50.870189 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:50.941782 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e9383-9827-443f-a70f-bf7250cfcc9d None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:51.014368 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:51.014979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:51.015284 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:51.015597 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:51.017223 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:51.094882 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:51.126194 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:51.127711 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a4529-27d8-4fc8-84ed-f82491d8fea6 service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:51.145979 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 503/1008] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:53.707985 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca705c-cdaa-4ba1-8528-e211d25e2d12 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:53.708508 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca705c-cdaa-4ba1-8528-e211d25e2d12 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:53.708592 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca705c-cdaa-4ba1-8528-e211d25e2d12 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:53.709897 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 506/1009] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:53.715801 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbbdf-a8c8-48be-aaf2-bd2f06027bb8 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:53.716203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbbdf-a8c8-48be-aaf2-bd2f06027bb8 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:53.716374 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbbdf-a8c8-48be-aaf2-bd2f06027bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:53.782232 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-e0ffbbdf-a8c8-48be-aaf2-bd2f06027bb8 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:53.783799 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-e0ffbbdf-a8c8-48be-aaf2-bd2f06027bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:53.879756 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 504/1010] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:54.158426 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca705c-cdaa-4ba1-8528-e211d25e2d12 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:54.160012 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-1bca705c-cdaa-4ba1-8528-e211d25e2d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:54.229682 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:54.262826 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca705c-cdaa-4ba1-8528-e211d25e2d12 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:54.336667 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:54.337288 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:54.337570 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:54.337842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:54.339436 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:54.446282 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:54.447717 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660c152-bb6a-434f-af9d-74657bf6374c service neutron] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:54.457570 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:54.465953 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 507/1011] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:52:56.641471 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c5284-b63a-4cee-b1e5-466e39f644b7 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:56.641966 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c5284-b63a-4cee-b1e5-466e39f644b7 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:56.642075 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c5284-b63a-4cee-b1e5-466e39f644b7 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:56.643389 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 505/1012] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:52:56.648690 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ece274-885d-412c-be96-a6413e320417 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:56.649081 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ece274-885d-412c-be96-a6413e320417 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:56.649478 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ece274-885d-412c-be96-a6413e320417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:56.715202 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-05ece274-885d-412c-be96-a6413e320417 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:52:56.716438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-05ece274-885d-412c-be96-a6413e320417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:56.812259 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 508/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:52:57.102719 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c5284-b63a-4cee-b1e5-466e39f644b7 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:52:57.104310 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-569c5284-b63a-4cee-b1e5-466e39f644b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:57.145041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:57.208003 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c5284-b63a-4cee-b1e5-466e39f644b7 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:52:57.280203 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:52:57.280800 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:52:57.281084 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:52:57.281363 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:52:57.282951 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:52:57.367785 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:52:57.393080 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:52:57.394503 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23431ada-ca82-4cee-92b3-62af10a9b5a3 service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:52:57.413034 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 506/1014] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:00.173856 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20180084-4f51-47e3-9054-4453cf80be2c None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:00.174375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20180084-4f51-47e3-9054-4453cf80be2c None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:00.174461 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20180084-4f51-47e3-9054-4453cf80be2c None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:00.175801 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 509/1015] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:00.181928 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a29e5-b3f5-4dc4-8bc1-6670de5445c1 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:00.182326 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a29e5-b3f5-4dc4-8bc1-6670de5445c1 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:00.182485 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a29e5-b3f5-4dc4-8bc1-6670de5445c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:00.248241 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-fb6a29e5-b3f5-4dc4-8bc1-6670de5445c1 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:00.249464 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-fb6a29e5-b3f5-4dc4-8bc1-6670de5445c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:00.344106 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 507/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:00.624221 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20180084-4f51-47e3-9054-4453cf80be2c None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:00.625761 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-20180084-4f51-47e3-9054-4453cf80be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:00.700758 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:00.728358 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20180084-4f51-47e3-9054-4453cf80be2c None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:00.803634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:00.804322 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:00.804607 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:00.804913 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:00.806563 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:00.915573 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:00.917803 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e947-5190-44ea-8fcf-3814e1a45aa0 service neutron] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:00.935863 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 510/1017] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:05.305647 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6429d-6741-4edf-9afa-1da98ad44fb7 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:05.306245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6429d-6741-4edf-9afa-1da98ad44fb7 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:05.306245 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6429d-6741-4edf-9afa-1da98ad44fb7 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:05.307727 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 508/1018] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:05.313164 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262f4a-5391-4691-bb5c-c11474312328 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:05.313598 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262f4a-5391-4691-bb5c-c11474312328 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:05.313849 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262f4a-5391-4691-bb5c-c11474312328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:05.319514 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:05.381664 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-01262f4a-5391-4691-bb5c-c11474312328 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:05.382883 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-01262f4a-5391-4691-bb5c-c11474312328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:05.482347 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 511/1019] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:05.763180 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6429d-6741-4edf-9afa-1da98ad44fb7 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:05.764863 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-11c6429d-6741-4edf-9afa-1da98ad44fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:05.816172 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:05.872504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6429d-6741-4edf-9afa-1da98ad44fb7 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:05.946757 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:05.948130 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:05.948130 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:05.948322 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:05.949898 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:06.047025 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:06.063537 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:06.064981 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3ed3f-1e04-4106-b2f0-6db600d4c591 service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:06.083287 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 509/1020] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:09.393008 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f13a05-e150-49cd-90c3-8ce0c1891dcb None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:09.393513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f13a05-e150-49cd-90c3-8ce0c1891dcb None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:09.393513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f13a05-e150-49cd-90c3-8ce0c1891dcb None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:09.394840 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 512/1021] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:09.400500 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61357837-d957-4b72-a908-2fa81af27c5c None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:09.400987 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61357837-d957-4b72-a908-2fa81af27c5c None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:09.401076 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61357837-d957-4b72-a908-2fa81af27c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:09.469650 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-61357837-d957-4b72-a908-2fa81af27c5c None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:09.470867 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-61357837-d957-4b72-a908-2fa81af27c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:09.572827 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 510/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:09.859367 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f13a05-e150-49cd-90c3-8ce0c1891dcb None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:09.860911 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-21f13a05-e150-49cd-90c3-8ce0c1891dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:09.971256 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:09.988748 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f13a05-e150-49cd-90c3-8ce0c1891dcb None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:10.069643 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:10.070268 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:10.070642 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:10.070929 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:10.073140 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:10.190671 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:10.192187 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a009e7-5bc0-461a-b32f-0fbf9600cedf service neutron] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:10.212001 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 513/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:14.656722 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba6641-1215-4c2f-918a-b2c6f7735c43 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:14.657217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba6641-1215-4c2f-918a-b2c6f7735c43 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:14.657312 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba6641-1215-4c2f-918a-b2c6f7735c43 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:14.658853 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 511/1024] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:53:14.664516 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022dc50-72fd-405a-8b99-0c5f84f4d917 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:14.665002 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022dc50-72fd-405a-8b99-0c5f84f4d917 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:14.665099 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022dc50-72fd-405a-8b99-0c5f84f4d917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:14.685513 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:14.746149 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-7022dc50-72fd-405a-8b99-0c5f84f4d917 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:14.747458 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-7022dc50-72fd-405a-8b99-0c5f84f4d917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:14.860754 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 514/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:53:15.148071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba6641-1215-4c2f-918a-b2c6f7735c43 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:15.149615 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-06ba6641-1215-4c2f-918a-b2c6f7735c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:15.207448 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:15.253353 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba6641-1215-4c2f-918a-b2c6f7735c43 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:15.326854 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f280acb87494dc59f0f7c222edcc87e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:15.327178 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:15.327523 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:15.327802 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:15.329408 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:15.435992 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:15.439473 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:15.440922 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d80c2-e4e6-4e3a-aa12-521526b0779a service neutron] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:15.458638 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 512/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:20.314331 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d99ce-b94d-404e-9093-7f3c50949ecb None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:20.314784 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d99ce-b94d-404e-9093-7f3c50949ecb None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:20.315145 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d99ce-b94d-404e-9093-7f3c50949ecb None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:20.316551 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 515/1027] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 05:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:20.326121 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51ef21-18e6-422f-b2b3-214300290d87 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:20.326557 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51ef21-18e6-422f-b2b3-214300290d87 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:20.326698 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51ef21-18e6-422f-b2b3-214300290d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:20.419225 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-4f51ef21-18e6-422f-b2b3-214300290d87 None None] MFA Rules not processed for user `c44be36d3a094a048ee37701ff21ed32`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:20.420993 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-4f51ef21-18e6-422f-b2b3-214300290d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:20.626962 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 513/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 05:53:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:24.666857 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbb8ac-90f1-4ffa-9609-40e6daed2c10 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:24.667586 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbb8ac-90f1-4ffa-9609-40e6daed2c10 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:24.667586 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbb8ac-90f1-4ffa-9609-40e6daed2c10 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:24.668963 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 516/1029] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 05:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:24.673662 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9bb5-7952-41e4-a788-93197da4215b None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:24.674105 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9bb5-7952-41e4-a788-93197da4215b None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:24.674308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c9bb5-7952-41e4-a788-93197da4215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:24.732209 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:24.755933 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-3c6c9bb5-7952-41e4-a788-93197da4215b None None] MFA Rules not processed for user `c44be36d3a094a048ee37701ff21ed32`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:24.757148 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-3c6c9bb5-7952-41e4-a788-93197da4215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:24.859173 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=59 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:24.885112 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 514/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 05:53:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:26.369496 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694a963-a5f5-45b7-bccc-76cbeb9b46bf None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:26.370035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694a963-a5f5-45b7-bccc-76cbeb9b46bf None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:26.370130 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694a963-a5f5-45b7-bccc-76cbeb9b46bf None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:26.371472 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 517/1031] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 05:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:26.376700 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7fdb4-8463-4ee2-b8b4-ebd633ac907a None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:26.377098 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7fdb4-8463-4ee2-b8b4-ebd633ac907a None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:26.377480 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7fdb4-8463-4ee2-b8b4-ebd633ac907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:26.458501 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-aad7fdb4-8463-4ee2-b8b4-ebd633ac907a None None] MFA Rules not processed for user `c44be36d3a094a048ee37701ff21ed32`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:26.459756 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-aad7fdb4-8463-4ee2-b8b4-ebd633ac907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:26.502210 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:26.569418 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 515/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 05:53:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:30.122787 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da64c-f8d8-4fc5-a233-23f2d653dc3d None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:30.123364 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da64c-f8d8-4fc5-a233-23f2d653dc3d None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:30.123364 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da64c-f8d8-4fc5-a233-23f2d653dc3d None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:30.125634 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 518/1033] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:30.131707 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ada56b-c90b-4d0c-a699-6e44c9e28ba0 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:30.132510 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ada56b-c90b-4d0c-a699-6e44c9e28ba0 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:30.133098 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ada56b-c90b-4d0c-a699-6e44c9e28ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:30.210639 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-38ada56b-c90b-4d0c-a699-6e44c9e28ba0 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:30.212472 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-38ada56b-c90b-4d0c-a699-6e44c9e28ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:30.310201 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 516/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 05:53:30.608160 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7f83e-b062-4f32-a92d-a75deea676f2 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:30.613474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7f83e-b062-4f32-a92d-a75deea676f2 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:30.613474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7f83e-b062-4f32-a92d-a75deea676f2 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:30.613474 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 519/1035] 10.222.0.13 () {54 vars in 961 bytes} [Mon Oct 17 05:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:30.619807 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981f339-a33e-4ed8-bf45-23f4ba7f65e3 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:30.620308 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981f339-a33e-4ed8-bf45-23f4ba7f65e3 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:30.620745 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981f339-a33e-4ed8-bf45-23f4ba7f65e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:30.643969 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:30.709917 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-0981f339-a33e-4ed8-bf45-23f4ba7f65e3 None None] MFA Rules not processed for user `921c6749f51c49d78748876ec65bd71f`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:30.711435 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-0981f339-a33e-4ed8-bf45-23f4ba7f65e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:30.843504 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 517/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Oct 17 05:53:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:30.851741 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7f83e-b062-4f32-a92d-a75deea676f2 None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:30.853368 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-76e7f83e-b062-4f32-a92d-a75deea676f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:31.046181 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7f83e-b062-4f32-a92d-a75deea676f2 None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:31.127705 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:31.169543 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '921c6749f51c49d78748876ec65bd71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:31.170273 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:31.170621 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:31.170961 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:31.172654 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:31.273842 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:31.276585 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42023f0-f757-46c2-a0f5-1a9ed3d383e9 service nova] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:31.297904 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 520/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:33.695450 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807efaf-199f-4974-9294-d97275e23860 None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:33.696542 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807efaf-199f-4974-9294-d97275e23860 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:33.696637 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807efaf-199f-4974-9294-d97275e23860 None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:33.697956 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 518/1038] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:33.703469 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618ae0-4342-426d-8bf4-497c7f7f6ac4 None None] REQUEST_METHOD: `POST` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:33.703897 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618ae0-4342-426d-8bf4-497c7f7f6ac4 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:33.704019 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618ae0-4342-426d-8bf4-497c7f7f6ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:33.870323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.auth.core [None req-f3618ae0-4342-426d-8bf4-497c7f7f6ac4 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:33.870323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-f3618ae0-4342-426d-8bf4-497c7f7f6ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:33.870323 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:33.875521 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 521/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:34.183776 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807efaf-199f-4974-9294-d97275e23860 None None] Authenticating user token {{(pid=153119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:34.185634 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-f807efaf-199f-4974-9294-d97275e23860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:34.271407 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:34.351041 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807efaf-199f-4974-9294-d97275e23860 None None] Validating token access rules against request {{(pid=153119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:34.471490 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '921c6749f51c49d78748876ec65bd71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:34.472120 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:34.472406 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:34.472687 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:34.474333 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:34.499136 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:34.571820 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:34.573484 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28bb21-4d26-4529-b330-2c5d789159de service nova] RBAC: Authorization granted {{(pid=153119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:34.593067 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 519/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 05:53:37.228291 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7291-e72e-4a12-8fdb-39a647d58dc8 None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:37.228821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7291-e72e-4a12-8fdb-39a647d58dc8 None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:37.228821 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7291-e72e-4a12-8fdb-39a647d58dc8 None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:37.230375 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 522/1041] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:37.235957 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82419289-71c2-4103-a630-c61506765155 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:37.236399 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82419289-71c2-4103-a630-c61506765155 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:37.236563 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82419289-71c2-4103-a630-c61506765155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:37.306949 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-82419289-71c2-4103-a630-c61506765155 None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:37.308217 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-82419289-71c2-4103-a630-c61506765155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:37.418164 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:37.426142 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 520/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:37.710966 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a3dc2-9745-4d59-9906-b0fca227b5fc None None] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:37.711965 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a3dc2-9745-4d59-9906-b0fca227b5fc None None] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:37.712438 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a3dc2-9745-4d59-9906-b0fca227b5fc None None] PATH_INFO: `/` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:37.714210 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 523/1043] 10.222.0.13 () {54 vars in 961 bytes} [Mon Oct 17 05:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:53:37.719813 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1309de-3cad-42f3-9106-15e2f7ebe1c6 None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:37.720107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1309de-3cad-42f3-9106-15e2f7ebe1c6 None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:37.729529 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1309de-3cad-42f3-9106-15e2f7ebe1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:37.806614 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-0d1309de-3cad-42f3-9106-15e2f7ebe1c6 None None] MFA Rules not processed for user `921c6749f51c49d78748876ec65bd71f`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:37.808141 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-0d1309de-3cad-42f3-9106-15e2f7ebe1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:37.937831 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 521/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Oct 17 05:53:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:37.946726 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a3dc2-9745-4d59-9906-b0fca227b5fc None None] Authenticating user token {{(pid=153118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:53:37.948252 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-937a3dc2-9745-4d59-9906-b0fca227b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:38.108528 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aebb8-3032-4199-aa96-6bc5f9d8aa8f None None] REQUEST_METHOD: `GET` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:38.108889 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aebb8-3032-4199-aa96-6bc5f9d8aa8f None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:38.109107 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aebb8-3032-4199-aa96-6bc5f9d8aa8f None None] PATH_INFO: `/` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:38.111348 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 522/1045] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:53:38.111848 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a3dc2-9745-4d59-9906-b0fca227b5fc None None] Validating token access rules against request {{(pid=153118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:53:38.115598 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a04764-153f-4799-be10-a489097e1adf None None] REQUEST_METHOD: `POST` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:38.115888 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a04764-153f-4799-be10-a489097e1adf None None] SCRIPT_NAME: `/identity` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:38.116167 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a04764-153f-4799-be10-a489097e1adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=153119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:38.136035 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=100 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:38.181238 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=100 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:38.189511 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.auth.core [None req-f9a04764-153f-4799-be10-a489097e1adf None None] MFA Rules not processed for user `437c2d620e4648bb9b558b4098744fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=153119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:53:38.191158 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG keystone.common.fernet_utils [None req-f9a04764-153f-4799-be10-a489097e1adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:38.238989 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '921c6749f51c49d78748876ec65bd71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e1d626bfda44be3804dced335b709b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=153118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:53:38.239604 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] REQUEST_METHOD: `GET` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:53:38.239845 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] SCRIPT_NAME: `/identity` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:53:38.240133 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=153118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:53:38.241785 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.fernet_utils [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=153118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:53:38.305195 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: [pid: 153119|app: 0|req: 523/1046] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 05:53:38.351624 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:53:38.354003 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b1d6d-9e0e-40e9-aadc-8bdb796dddb0 service nova] RBAC: Authorization granted {{(pid=153118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:53:38.373652 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: [pid: 153118|app: 0|req: 524/1047] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 05:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 05:53:48.303665 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: DEBUG dbcounter [-] [153119] Writing DB stats keystone:SELECT=44 {{(pid=153119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:48.371049 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: DEBUG dbcounter [-] [153118] Writing DB stats keystone:SELECT=89 {{(pid=153118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:53:58.756859 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: Mon Oct 17 05:53:58 2022 - graceful shutdown triggered... Oct 17 05:53:58.757325 dv-cinder-861566-1-iscsi devstack@keystone.service[153118]: Gracefully killing worker 1 (pid: 153118)... Oct 17 05:53:58.757693 dv-cinder-861566-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Oct 17 05:53:58.758071 dv-cinder-861566-1-iscsi devstack@keystone.service[153119]: Gracefully killing worker 2 (pid: 153119)... Oct 17 05:53:59.757986 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: worker 1 buried after 1 seconds Oct 17 05:53:59.758365 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: worker 2 buried after 1 seconds Oct 17 05:53:59.758365 dv-cinder-861566-1-iscsi devstack@keystone.service[153106]: goodbye to uWSGI. Oct 17 05:53:59.759571 dv-cinder-861566-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Oct 17 05:53:59.760163 dv-cinder-861566-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. Oct 17 05:59:21.151635 dv-cinder-861566-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 05:59:21.160835 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 05:59:21.163592 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 05:59:21 2022] *** Oct 17 05:59:21.163814 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 05:59:21.163980 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 05:59:21.164135 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: nodename: dv-cinder-861566-1-iscsi Oct 17 05:59:21.164279 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: machine: x86_64 Oct 17 05:59:21.164415 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: clock source: unix Oct 17 05:59:21.164541 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: pcre jit disabled Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: detected number of CPU cores: 4 Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: current working directory: / Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: detected binary path: /usr/bin/uwsgi-core Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: your processes number limit is 47680 Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: your memory page size is 4096 bytes Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: detected max file descriptor number: 2048 Oct 17 05:59:21.164693 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: lock engine: pthread robust mutexes Oct 17 05:59:21.165190 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: thunder lock: enabled Oct 17 05:59:21.165190 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 05:59:21.165190 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 05:59:21.203561 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: Python main interpreter initialized at 0x558c6dc3a9c0 Oct 17 05:59:21.203561 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: python threads support enabled Oct 17 05:59:21.203561 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: your server socket listen backlog is limited to 100 connections Oct 17 05:59:21.203561 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: your mercy for graceful operations on workers is 90 seconds Oct 17 05:59:21.203940 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 05:59:21.204021 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: *** Operational MODE: preforking *** Oct 17 05:59:21.204234 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: *** uWSGI is running in multiple interpreter mode *** Oct 17 05:59:21.204234 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: spawned uWSGI master process (pid: 218489) Oct 17 05:59:21.204928 dv-cinder-861566-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 05:59:21.206075 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: spawned uWSGI worker 1 (pid: 218501, cores: 1) Oct 17 05:59:21.206075 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: spawned uWSGI worker 2 (pid: 218502, cores: 1) Oct 17 05:59:21.206075 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 05:59:25.515510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:59:25.515510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: from cryptography.utils import int_from_bytes Oct 17 05:59:25.516312 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:59:25.516312 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: from cryptography.utils import int_from_bytes Oct 17 05:59:25.516649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:59:25.516649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: from cryptography.utils import int_from_bytes Oct 17 05:59:25.517613 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 05:59:25.517613 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: from cryptography.utils import int_from_bytes Oct 17 05:59:25.566253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 05:59:25.566425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 05:59:25.566624 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] command line args: [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 05:59:25.566842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 05:59:25.566939 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 05:59:25.567027 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 05:59:25.567114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ================================================================================ {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 05:59:25.567219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] command line args: [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 05:59:25.567354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] admin_token = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.567498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 05:59:25.567595 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] config_dir = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.567682 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ================================================================================ {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 05:59:25.567768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.567900 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] config_source = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.567995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] admin_token = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568082 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] debug = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] config_dir = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] config_source = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568826 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.568941 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] debug = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] insecure_debug = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569175 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569321 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569437 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569437 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] insecure_debug = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569655 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569770 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.569891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_config_append = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570105 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_dir = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570561 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_config_append = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_file = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_dir = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.570910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_file = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571549 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571549 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.571949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572425 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572595 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572686 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572796 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.572966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573134 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573219 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573358 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573458 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] notification_format = cadf {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573636 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573721 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] notification_format = cadf {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.573968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574116 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] publish_errors = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574352 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574443 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574536 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] publish_errors = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574752 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574839 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.574984 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575078 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575164 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575331 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575436 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] standard_threads = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] strict_password_check = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575844 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] standard_threads = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.575930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] strict_password_check = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] use_eventlog = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576244 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] use_journal = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576327 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576418 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] use_json = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] use_eventlog = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] use_stderr = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576675 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] use_journal = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] use_syslog = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576844 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] use_json = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.576939 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] watch_log_file = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.577087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] use_stderr = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.577177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] use_syslog = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.577261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.577346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] watch_log_file = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 05:59:25.577444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.577572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.577673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.577810 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.577917 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578065 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578302 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578643 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.578889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.579034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.579126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.579815 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.579921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580069 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580302 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580396 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580785 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.580883 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581031 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581125 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581513 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581756 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.581992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582722 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582822 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.582958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583064 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583486 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583583 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583721 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.583952 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584275 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584513 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.584970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585108 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585196 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585339 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585802 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.585890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586179 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586276 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586530 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586618 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.586944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587531 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.587987 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588221 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588361 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588828 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.588920 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589058 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589149 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589384 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589752 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.589979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590215 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590303 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590765 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.590996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591227 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591720 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.591958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592193 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592422 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592664 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.592884 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593266 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593359 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593827 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.593964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594306 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594691 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.594992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.595620 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.595736 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.595836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.595970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.enabled = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596298 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596535 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.enabled = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596673 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596765 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.596906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597003 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597145 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597378 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597465 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.597938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598077 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598164 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598408 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598545 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.598867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599031 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599268 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599437 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599610 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.599993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600318 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600774 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.600918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601006 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601147 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601245 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601604 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601697 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.601933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602160 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602306 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602393 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602637 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602782 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.602880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603120 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603416 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603506 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.external = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603744 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.mapped = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.603992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.external = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604077 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604223 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.mapped = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604310 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.password = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604567 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] auth.token = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.604964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605053 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.password = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605171 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] catalog.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605264 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] auth.token = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605407 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605721 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] catalog.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605880 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.605971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606110 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.606904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607452 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607637 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607782 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.607870 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608109 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608255 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608352 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608734 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.608985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609082 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609222 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609568 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.609987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610083 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610736 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610831 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.610983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611076 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611223 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611380 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611716 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611805 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.611949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612045 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612196 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612290 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612678 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.612913 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613158 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613636 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.613961 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614104 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614659 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614803 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.614899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615135 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615419 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.615995 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616685 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.616928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617017 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617266 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.617990 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618234 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618372 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618466 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618712 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.618924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619156 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619330 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619553 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619792 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.619887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620112 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620264 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620355 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620729 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620815 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.620972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621303 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621438 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621675 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621900 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.621987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.password = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622225 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622360 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.password = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622826 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.622916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623154 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623310 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.623996 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624249 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624380 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624854 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.624948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625416 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625644 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.625881 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626022 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626531 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626628 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.626857 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627102 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627854 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.627963 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628660 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628889 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.628984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629375 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629472 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629618 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.629934 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630091 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630414 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630645 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.630880 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631750 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.631996 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632088 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] policy.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632223 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632314 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632569 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] policy.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632800 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.632926 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] receipt.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] receipt.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633539 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633839 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.633932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634301 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634536 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634776 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.634862 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635106 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635360 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635763 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] revoke.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.635868 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636245 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636345 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] revoke.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] role.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] role.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636810 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.636951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] role.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637043 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] role.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637179 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] role.list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] role.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637423 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] role.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637509 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] role.list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.637986 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638229 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638385 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638477 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638627 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638721 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.638964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639110 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639197 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639363 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639707 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.639934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640083 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640325 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640412 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640567 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640720 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640819 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.640961 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.641194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.641281 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.641432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.641525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.641688 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.642269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.642432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.642527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.642666 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.642763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.642916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643235 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643414 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643647 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643879 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.643967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644107 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644339 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644568 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644655 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.644912 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645285 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645610 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.645975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.provider = fernet {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646689 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.provider = fernet {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.646924 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647024 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647159 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647522 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.647994 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] trust.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648228 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] trust.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648613 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648700 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648849 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.648935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649082 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649178 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649327 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649669 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.649908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.connection = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650312 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650404 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650559 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650646 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.connection = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.650891 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651036 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651124 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651621 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651841 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.651987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652221 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652315 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652452 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652583 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652757 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.652914 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653003 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653155 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653858 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.653945 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654097 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654419 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 05:59:25.654578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 05:59:25.654957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=218502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 05:59:25.683897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.684128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.684354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.684667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.684899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.685125 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.692722 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.693484 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.721445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.721616 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.722245 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.722478 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.773735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.774133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.822416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.822643 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.823265 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.823521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.978733 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.978919 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979118 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979603 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979691 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979780 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:25.979896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.002551 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.002727 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.002822 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.002961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003186 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003641 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.003955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.004092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.014784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.015108 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.027175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.027430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.027601 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.027691 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.027831 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.027920 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028266 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028363 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028476 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218501) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.028883 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=218502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 05:59:26.045401 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.045611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:59:26.045737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.045949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:59:26.046039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.046148 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:59:26.046290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.046422 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.046512 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 05:59:26.046686 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.046774 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.046948 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.047037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.047209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.047338 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.047546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.057117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.057399 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.057500 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.057672 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.057775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.057954 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.058050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.058276 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.065481 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.065778 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.068405 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.068696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.068785 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.068983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069815 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.069909 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070359 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070446 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070778 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.070905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.071013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.071170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.071269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.071461 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.093732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.094143 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.095760 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.096047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.096183 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.096471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.100696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.100984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.101148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.101252 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.101437 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.101720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.107603 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.107893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.108081 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.108370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.113515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.113814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.114109 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114303 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114878 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.114976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.115160 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.115257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.115463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.115554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.115736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.116006 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.131503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.131782 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.131965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.132246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.135652 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.136112 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.138882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.139181 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.139380 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.139682 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.143451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.143978 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.146636 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.147226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.149925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.150228 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.150456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.150562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.150757 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.150867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.151046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.151137 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.151351 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.151444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.151636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.151916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.167677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.167951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.168043 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.168221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.168323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.168486 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.168576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.168756 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.175395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.175617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.176954 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.177132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.178477 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.178630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.178757 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.178915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.181774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.181995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.182099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.182290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.182381 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.182573 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.182667 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.182861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.182995 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.183245 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.194654 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.194951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.195040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.195237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.195363 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.195548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.195638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.195836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.195932 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.196213 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.196310 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.196491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.196579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.196781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.211901 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.212516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.214810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.215442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.217712 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.218330 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.219253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.220392 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.221410 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.221700 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.221862 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.222105 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.222195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.222540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.228501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.228970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.229962 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.230385 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.232860 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.233261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.233359 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.233549 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.233718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.233810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.233956 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.234209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.241677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.241976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.242137 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.242263 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.242431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.242712 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.249166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=218501) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.249453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.249602 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=218502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 05:59:26.249735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.249893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250171 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250444 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250533 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250824 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.250998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.251592 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.251810 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.251916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252204 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252472 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.252913 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.253048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.253174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.253257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=218501) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.253444 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.253720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=218502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 05:59:26.286848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558c6dc3a9c0 pid: 218501 (default app) Oct 17 05:59:26.288080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558c6dc3a9c0 pid: 218502 (default app) Oct 17 05:59:26.290146 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79afaea-6f92-4b4b-abad-f4c845b995ed None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:26.290436 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79afaea-6f92-4b4b-abad-f4c845b995ed None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:26.290711 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79afaea-6f92-4b4b-abad-f4c845b995ed None None] PATH_INFO: `/v3` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:26.294869 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon Oct 17 05:59:26 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:33.739069 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:33.739626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:33.739626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:33.743859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:59:33.749298 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:33.749732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:33.749836 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:33.822101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: INFO dbcounter [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] Registered counter for database keystone Oct 17 05:59:33.842006 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG oslo_db.sqlalchemy.engines [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=218501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 05:59:33.848292 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writer thread running {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 05:59:34.028583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG passlib.handlers.bcrypt [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=218501) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 05:59:34.028885 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG passlib.handlers.bcrypt [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=218501) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 05:59:34.086975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:34.088645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:34.214648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:59:34.481989 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:34.552316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: INFO dbcounter [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] Registered counter for database keystone Oct 17 05:59:34.573694 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG oslo_db.sqlalchemy.engines [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=218502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 05:59:34.580572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writer thread running {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 05:59:34.698590 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bd262-20d3-441c-bdd8-e15290d2bad9 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:34.702918 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:34.703579 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:34.703865 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:34.704653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:34.712812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:34.763842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: WARNING keystone.common.rbac_enforcer.enforcer [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 05:59:34.764377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a3589-5a84-4ca2-8641-8beb5e1b0e96 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:34.773921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Oct 17 05:59:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 05:59:34.779963 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:34.797545 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6069501-2e8b-4d7a-a944-aeed1c8b181d None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:34.802136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:34.803252 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:34.803566 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:34.803841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] PATH_INFO: `/v3/projects/67b5d25652604b029091af61167fe6bf` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:34.805443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] RBAC: Authorizing `identity:get_project(project_id=67b5d25652604b029091af61167fe6bf)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:34.857708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: WARNING keystone.common.rbac_enforcer.enforcer [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 05:59:34.858057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff1f14-091b-4620-81c9-4e68767f41aa admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:34.862585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 05:59:34 2022] GET /identity/v3/projects/67b5d25652604b029091af61167fe6bf => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:36.982516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440457c-5613-4d3a-b722-2f7b24ae3f86 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:36.983078 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440457c-5613-4d3a-b722-2f7b24ae3f86 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:36.983078 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440457c-5613-4d3a-b722-2f7b24ae3f86 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:36.987744 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:36.993153 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:36.993535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:36.993705 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:37.014150 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:37.015688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:37.023562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 05:59:37.289439 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440457c-5613-4d3a-b722-2f7b24ae3f86 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:37.302090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440457c-5613-4d3a-b722-2f7b24ae3f86 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:37.306574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef52b4-2b56-4037-bc22-97ebf665c994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:37.307175 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef52b4-2b56-4037-bc22-97ebf665c994 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:37.307486 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef52b4-2b56-4037-bc22-97ebf665c994 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:37.307761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef52b4-2b56-4037-bc22-97ebf665c994 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:37.315841 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef52b4-2b56-4037-bc22-97ebf665c994 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:37.317600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef52b4-2b56-4037-bc22-97ebf665c994 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:37.326235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Oct 17 05:59:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 05:59:37.332050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:37.341055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8e30-7071-48c8-bfb8-8919aabb16f0 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:37.345667 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee8c00-f8af-4c4d-9252-375019473e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:37.346301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee8c00-f8af-4c4d-9252-375019473e91 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:37.346584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee8c00-f8af-4c4d-9252-375019473e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:37.346861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee8c00-f8af-4c4d-9252-375019473e91 admin admin] PATH_INFO: `/v3/users/bf714812a1a846729a9dd3749da0e163` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:37.348228 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee8c00-f8af-4c4d-9252-375019473e91 admin admin] RBAC: Authorizing `identity:get_user(user_id=bf714812a1a846729a9dd3749da0e163)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:37.350056 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee8c00-f8af-4c4d-9252-375019473e91 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:37.351622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon Oct 17 05:59:37 2022] GET /identity/v3/users/bf714812a1a846729a9dd3749da0e163 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:39.520210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4b3f-2ccf-4e4e-8429-f7f140fed320 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:39.520833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4b3f-2ccf-4e4e-8429-f7f140fed320 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:39.520833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4b3f-2ccf-4e4e-8429-f7f140fed320 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:39.522064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:39.527215 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:39.527680 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:39.527918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:39.546241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:39.547457 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:39.596329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c40579-fe8a-446e-8773-034f46c3b5cf None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:39.596614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c40579-fe8a-446e-8773-034f46c3b5cf None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:39.596892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c40579-fe8a-446e-8773-034f46c3b5cf None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:39.598215 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:59:39.603257 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f226c-7638-4e03-a62e-22817923b452 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:39.603580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f226c-7638-4e03-a62e-22817923b452 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:39.603857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f226c-7638-4e03-a62e-22817923b452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:39.628350 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:39.645661 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG passlib.handlers.bcrypt [None req-667f226c-7638-4e03-a62e-22817923b452 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=218502) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 05:59:39.646056 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG passlib.handlers.bcrypt [None req-667f226c-7638-4e03-a62e-22817923b452 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=218502) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 05:59:39.695490 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-667f226c-7638-4e03-a62e-22817923b452 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:39.697063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-667f226c-7638-4e03-a62e-22817923b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:39.794431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 05:59:39.897137 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:39.921762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:39.999788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687467f-ba2e-405c-99a3-10d96ebf7d08 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.033140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.033768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.034049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.034323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.045308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:59:40.054548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:40.056023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.056990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:59:40.061798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f226c-7638-4e03-a62e-22817923b452 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.062085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.146060 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:40.163915 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088b1b5-e228-4f0b-aeea-153f06b2bbc4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.164353 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f226c-7638-4e03-a62e-22817923b452 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.166700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:40.198624 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.199236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.199544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.199820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.200492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.201885 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.203068 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.204073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.204711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.204993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.205850 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:40.206158 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.207620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.220383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:40.226381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.237599 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.sql.core [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:59:40.237599 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:59:40.237599 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [parameters: {'id': '77ac398c8ece42b69cfd0394fb9c9639', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:59:40.237599 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:59:40.254583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result.read() Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 05:59:40.255629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application yield resource Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application next(self.gen) Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise self.value Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application session.commit() Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 05:59:40.256679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.session.flush() Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application compat.raise_( Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 05:59:40.257698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 05:59:40.258708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application conn.query(q) Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result.read() Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 05:59:40.259646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [parameters: {'id': '77ac398c8ece42b69cfd0394fb9c9639', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 05:59:40.260556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:40.261510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 8/16] 10.222.0.13 () {60 vars in 1137 bytes} [Mon Oct 17 05:59:40 2022] POST /identity/v3/domains => generated 190 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:59:40.262439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.326157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b251c5-0f8a-4285-8fbc-38c9d8a0c3ec None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.359705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.361122 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.361404 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.361678 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] PATH_INFO: `/v3/users/bf714812a1a846729a9dd3749da0e163` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.362737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87e876-caa7-4b8f-843c-01aa0ff2e0b7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.363202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] RBAC: Authorizing `identity:get_user(user_id=bf714812a1a846729a9dd3749da0e163)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.364675 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.366266 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:59:40 2022] GET /identity/v3/users/bf714812a1a846729a9dd3749da0e163 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:40.372005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.398395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.399020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.399311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.399598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.405280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:40.410045 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.411533 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.418804 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:59:40.427604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:40.428754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:59:40.429856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:59:40.429856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:59:40.429856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:40.429856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:59:40.434660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.477901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b3f3a-66de-4e24-bfcf-34f752cd1d67 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.511869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.512511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.512794 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.513071 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.514278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.515965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.517535 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:40.523131 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:40.536236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5403de0-e07b-4481-9f39-ede8a21f35e3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.570550 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.574579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.574579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.574579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.574579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:40.574579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.574579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2458-f02d-4c1c-8412-903e9779eea2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.591399 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:40.625014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf2c9-375e-4299-9f02-c7a12d9018bc None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:40.627352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:40.663171 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aca7d-0394-4c1c-8b46-36e6f1d3a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:40.663888 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aca7d-0394-4c1c-8b46-36e6f1d3a9d0 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:40.664170 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aca7d-0394-4c1c-8b46-36e6f1d3a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:40.664448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aca7d-0394-4c1c-8b46-36e6f1d3a9d0 None admin] PATH_INFO: `/v3/domains/default/users/bf714812a1a846729a9dd3749da0e163/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:40.666509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448aca7d-0394-4c1c-8b46-36e6f1d3a9d0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bf714812a1a846729a9dd3749da0e163, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:40.668030 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448aca7d-0394-4c1c-8b46-36e6f1d3a9d0 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:40.853676 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 11/21] 10.222.0.13 () {58 vars in 1361 bytes} [Mon Oct 17 05:59:40 2022] PUT /identity/v3/domains/default/users/bf714812a1a846729a9dd3749da0e163/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 05:59:43.319535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733889c-b324-464a-89a4-419ea2b51235 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.320086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733889c-b324-464a-89a4-419ea2b51235 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.320086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733889c-b324-464a-89a4-419ea2b51235 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.321570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:43.327638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27712f37-aa10-4d8e-8a99-d7a7e26dfb93 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.328456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27712f37-aa10-4d8e-8a99-d7a7e26dfb93 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.329014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27712f37-aa10-4d8e-8a99-d7a7e26dfb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.357023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-27712f37-aa10-4d8e-8a99-d7a7e26dfb93 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:43.358897 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-27712f37-aa10-4d8e-8a99-d7a7e26dfb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:43.454166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:43.587124 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6a75b-e13a-4743-bb8e-539922b309da None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.587501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6a75b-e13a-4743-bb8e-539922b309da None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.587735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6a75b-e13a-4743-bb8e-539922b309da None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.589464 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:43.595846 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dacd7-b4f0-472e-9118-6b7e4e625717 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.596180 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dacd7-b4f0-472e-9118-6b7e4e625717 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.596407 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dacd7-b4f0-472e-9118-6b7e4e625717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.623469 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-1b2dacd7-b4f0-472e-9118-6b7e4e625717 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:43.624679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-1b2dacd7-b4f0-472e-9118-6b7e4e625717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:43.705590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4786c-98f7-4f17-9732-0ba26e51a66d None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.705927 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4786c-98f7-4f17-9732-0ba26e51a66d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.706156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4786c-98f7-4f17-9732-0ba26e51a66d None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.711329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 13/25] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 05:59:43.717292 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fab6b-0f26-4660-a2a9-a6a7dc5e7466 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.717583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fab6b-0f26-4660-a2a9-a6a7dc5e7466 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.717759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 13/26] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:43.717865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fab6b-0f26-4660-a2a9-a6a7dc5e7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.755567 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-e06fab6b-0f26-4660-a2a9-a6a7dc5e7466 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:43.756872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-e06fab6b-0f26-4660-a2a9-a6a7dc5e7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:43.864367 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:43.885021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 14/27] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:43.950892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a676-fd0a-4a89-bba1-f61a02a88e2a None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.951216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a676-fd0a-4a89-bba1-f61a02a88e2a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.951479 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a676-fd0a-4a89-bba1-f61a02a88e2a None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.956018 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 14/28] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:43.961600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:43.961895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:43.962177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:43.983768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:43.985057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:44.004138 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a676-fd0a-4a89-bba1-f61a02a88e2a None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:44.046894 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:44.113742 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 15/29] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:44.200586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a676-fd0a-4a89-bba1-f61a02a88e2a None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:44.222842 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:44.246123 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:44.246909 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.247193 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.247491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.269210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:44.271162 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1edd85e2-ff54-44bf-adf8-801790e48dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 05:59:44.293146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:44.294146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 05:59:44.295048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 05:59:44.295048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 05:59:44.295048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 05:59:44.295048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 05:59:44.295757 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaac2c-4571-4a9a-984f-0b496f8f8425 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.296045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaac2c-4571-4a9a-984f-0b496f8f8425 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.296324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaac2c-4571-4a9a-984f-0b496f8f8425 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.299202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 16/31] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:44.300325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeaac2c-4571-4a9a-984f-0b496f8f8425 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:44.393607 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:44.411411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cbd3c-da09-4e7a-994f-7cc34fa47472 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:44.423497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeaac2c-4571-4a9a-984f-0b496f8f8425 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:44.444432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:44.458175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:44.459091 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.459409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.462415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.462415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:44.462415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:44.464190 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:44.464888 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.465287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.465648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.466379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:44.466750 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:44.468452 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb039a1-3af9-46bb-9773-cf355d97b8d5 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:44.488497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 17/32] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 05:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:44.489843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.490195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.490540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.509641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:44.510974 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:44.582623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 18/33] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:44.588282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:44.657668 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 16/34] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:59:44 2022] POST /identity/v3/roles => generated 219 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:59:44.659044 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:44.665098 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:44.708615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822947c7-3bd7-4ee6-9404-48659e3d9b3a None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:44.751853 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:44.752886 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.753408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.753919 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.754872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:44.757090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:44.851517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109176-7e79-4e3a-a3e6-59bf89ac9bcb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:44.869240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 19/35] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:59:44 2022] POST /identity/v3/roles => generated 225 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:59:44.869945 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:44.896563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:44.897199 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:44.897532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:44.897814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:44.898485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:44.900161 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.012415 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a02808-7bab-46c4-a388-61fbd4efc4b6 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.016928 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 17/36] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:59:44 2022] POST /identity/v3/projects => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:59:45.017919 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.052206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:45.071118 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8734e-46e6-43da-8ba5-ce9bc6f42d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.072054 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8734e-46e6-43da-8ba5-ce9bc6f42d69 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.072181 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8734e-46e6-43da-8ba5-ce9bc6f42d69 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.072374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8734e-46e6-43da-8ba5-ce9bc6f42d69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.076413 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:45.094973 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8734e-46e6-43da-8ba5-ce9bc6f42d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-81b8734e-46e6-43da-8ba5-ce9bc6f42d69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:59:45.101643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:45.102624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 05:59:45.103502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 20/37] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:59:45.108801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010775-4f76-4703-b531-b3a8427c183a None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.109115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010775-4f76-4703-b531-b3a8427c183a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.109367 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010775-4f76-4703-b531-b3a8427c183a None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.112983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 21/38] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:45.114349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010775-4f76-4703-b531-b3a8427c183a None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.176754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca580f-8f3d-4566-bf60-4e219fe34629 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.217640 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.218273 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.218564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.218845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.220251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.222795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.251949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010775-4f76-4703-b531-b3a8427c183a None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.287834 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.288382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.288662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.288934 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.289610 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:45.289953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.291372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470add9-4ade-44a6-98e5-e84a8b6511e3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.300665 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 22/39] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:45.304568 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 18/40] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 05:59:45 2022] POST /identity/v3/roles => generated 223 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:59:45.307678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.308441 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.308734 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.309011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.329768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:45.333395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:45.381074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:45.413786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 23/41] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:45.416816 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:45.426879 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eabf75-88fb-43c3-81ba-e6e952e51b40 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.571662 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.572230 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.572513 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.572789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] PATH_INFO: `/v3/users/bf714812a1a846729a9dd3749da0e163` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.573966 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] RBAC: Authorizing `identity:get_user(user_id=bf714812a1a846729a9dd3749da0e163)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.575755 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.577648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 19/42] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:59:45 2022] GET /identity/v3/users/bf714812a1a846729a9dd3749da0e163 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:45.584520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.682586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf85f1c-32fc-4d95-967a-eb3eada4784a None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.689255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.718050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.718695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.718984 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.719347 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.720476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.721930 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.723551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 24/43] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:45.729404 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.774733 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:45.789597 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504fa17-a709-42dd-ad9c-55f99aff1e68 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.824565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.825139 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.825431 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.825716 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.826874 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.828376 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.828545 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e1655-9325-469f-9f4e-45595769f142 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.829988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 20/44] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:45.835635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:45.862532 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.863147 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.863461 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.863743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.864040 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:45.864466 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.domain.id=default, user.id=bf714812a1a846729a9dd3749da0e163) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:45.864777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.866212 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7326b54-2d60-4028-95b9-2ac93c57bd28 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:45.874516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Oct 17 05:59:45 2022] GET /identity/v3/role_assignments?user.id=bf714812a1a846729a9dd3749da0e163&scope.domain.id=default&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 05:59:45.948719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f20b-fc3b-4abe-85f0-c20a2a74128c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:45.991167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e7c72-cc45-43c7-aa99-872c08debfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:45.991841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e7c72-cc45-43c7-aa99-872c08debfbf None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:45.992126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e7c72-cc45-43c7-aa99-872c08debfbf None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:45.992405 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e7c72-cc45-43c7-aa99-872c08debfbf None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:45.993078 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e7c72-cc45-43c7-aa99-872c08debfbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:45.994575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e7c72-cc45-43c7-aa99-872c08debfbf None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:46.097731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 21/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 05:59:45 2022] POST /identity/v3/projects => generated 312 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 05:59:48.212915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112752ca-dda9-40ad-993a-415c18bbd842 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:48.213473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112752ca-dda9-40ad-993a-415c18bbd842 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:48.213473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112752ca-dda9-40ad-993a-415c18bbd842 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:48.214802 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:48.220345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:48.220783 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:48.220907 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:48.239724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:48.241251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:48.306595 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:48.572284 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112752ca-dda9-40ad-993a-415c18bbd842 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:48.670395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112752ca-dda9-40ad-993a-415c18bbd842 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:48.679556 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:48.702986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d60022-d7b5-4107-ae41-e04b82a1a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:48.703671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60022-d7b5-4107-ae41-e04b82a1a20f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:48.703953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60022-d7b5-4107-ae41-e04b82a1a20f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:48.704229 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60022-d7b5-4107-ae41-e04b82a1a20f None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:48.705340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d60022-d7b5-4107-ae41-e04b82a1a20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:48.706790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d60022-d7b5-4107-ae41-e04b82a1a20f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:48.708368 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 27/49] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:48.713889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:48.724527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:48.812311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033e7e8-be6d-40f0-8a63-27f976ffcb7d None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:48.845363 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e658be-113f-4f9d-b7a6-f9d7ba400fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:48.845968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e658be-113f-4f9d-b7a6-f9d7ba400fb6 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:48.846254 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e658be-113f-4f9d-b7a6-f9d7ba400fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:48.846532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e658be-113f-4f9d-b7a6-f9d7ba400fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:48.847192 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e658be-113f-4f9d-b7a6-f9d7ba400fb6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:48.848680 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e658be-113f-4f9d-b7a6-f9d7ba400fb6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:48.929223 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 23/50] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Oct 17 05:59:48 2022] POST /identity/v3/projects => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:59:51.058980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac4c62-c486-4ba9-ba97-4db3a990bd36 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:51.059560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac4c62-c486-4ba9-ba97-4db3a990bd36 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:51.059560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac4c62-c486-4ba9-ba97-4db3a990bd36 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:51.060887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 28/51] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:51.066603 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:51.066995 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:51.067170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:51.084929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:51.086120 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:51.139555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:51.151416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 24/52] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:51.420295 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac4c62-c486-4ba9-ba97-4db3a990bd36 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:51.520813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac4c62-c486-4ba9-ba97-4db3a990bd36 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:51.552332 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfccf4a-e7d5-4007-81f0-57148dee0df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:51.552957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfccf4a-e7d5-4007-81f0-57148dee0df7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:51.553237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfccf4a-e7d5-4007-81f0-57148dee0df7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:51.553512 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfccf4a-e7d5-4007-81f0-57148dee0df7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:51.554869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfccf4a-e7d5-4007-81f0-57148dee0df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:51.556597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfccf4a-e7d5-4007-81f0-57148dee0df7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:51.558148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 29/53] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:51.564158 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:51.660442 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2325417-967d-46f1-9abe-b8da012cb021 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:51.694058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5fd6b-006c-4268-825b-80359c747099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:51.694656 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5fd6b-006c-4268-825b-80359c747099 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:51.694946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5fd6b-006c-4268-825b-80359c747099 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:51.695228 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5fd6b-006c-4268-825b-80359c747099 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:51.695926 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5fd6b-006c-4268-825b-80359c747099 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:51.697376 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5fd6b-006c-4268-825b-80359c747099 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:51.768740 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 25/54] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:59:51 2022] POST /identity/v3/users => generated 275 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:59:55.029369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce951d2-9af4-4149-8135-d5f8819070a2 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.029910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce951d2-9af4-4149-8135-d5f8819070a2 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.029910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce951d2-9af4-4149-8135-d5f8819070a2 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.031375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 30/55] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:55.036623 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d5c6e-fb5f-41ae-85b0-06ab0a1b0856 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.037527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d5c6e-fb5f-41ae-85b0-06ab0a1b0856 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.038020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d5c6e-fb5f-41ae-85b0-06ab0a1b0856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.064516 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-8d2d5c6e-fb5f-41ae-85b0-06ab0a1b0856 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:55.066172 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-8d2d5c6e-fb5f-41ae-85b0-06ab0a1b0856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:55.130093 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:55.165824 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 26/56] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:55.195478 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4dd9f-ce9e-45a0-9caa-e8fcf1d6826d None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.196146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4dd9f-ce9e-45a0-9caa-e8fcf1d6826d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.196639 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4dd9f-ce9e-45a0-9caa-e8fcf1d6826d None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.198202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 31/57] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:55.204079 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8ab72-4f98-47e4-ac7e-d1e91d3072ea None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.204372 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8ab72-4f98-47e4-ac7e-d1e91d3072ea None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.204841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8ab72-4f98-47e4-ac7e-d1e91d3072ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.230481 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-09d8ab72-4f98-47e4-ac7e-d1e91d3072ea None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:55.232250 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-09d8ab72-4f98-47e4-ac7e-d1e91d3072ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:55.318661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 27/58] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:55.350812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a266bd-1b7c-4288-a5c9-f3adf70b6eeb None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.351260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a266bd-1b7c-4288-a5c9-f3adf70b6eeb None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.351688 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a266bd-1b7c-4288-a5c9-f3adf70b6eeb None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.353144 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 32/59] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:55.364627 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b4773-8440-42ed-9c40-2bbb8fe957a8 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.365204 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b4773-8440-42ed-9c40-2bbb8fe957a8 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.365719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b4773-8440-42ed-9c40-2bbb8fe957a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.387062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-302b4773-8440-42ed-9c40-2bbb8fe957a8 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:55.388634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-302b4773-8440-42ed-9c40-2bbb8fe957a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:55.473932 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 28/60] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:55.565128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a266bd-1b7c-4288-a5c9-f3adf70b6eeb None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:55.570636 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b4488-23dd-457e-af18-e021fab60b01 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.571169 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b4488-23dd-457e-af18-e021fab60b01 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.571704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b4488-23dd-457e-af18-e021fab60b01 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.573265 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 29/61] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:55.578868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.579433 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.579959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.613054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:55.615984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:55.651463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:55.661846 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:55.713060 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 30/62] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:55.715186 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:55.733595 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a266bd-1b7c-4288-a5c9-f3adf70b6eeb None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:55.775832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:55.776483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.776769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.777047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.788638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:59:55.869084 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed5868-7ac7-4a8a-85de-5339145380dc None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:59:55.901505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:55.902630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 33/63] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:59:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:59:55.903912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:55.911002 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee15157-05cb-48d0-9006-16378c600b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:55.911685 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15157-05cb-48d0-9006-16378c600b2a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.911975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15157-05cb-48d0-9006-16378c600b2a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.912256 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15157-05cb-48d0-9006-16378c600b2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.922981 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee15157-05cb-48d0-9006-16378c600b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-bee15157-05cb-48d0-9006-16378c600b2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:59:55.928012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:55.928974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:55.929843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 31/64] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 05:59:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:59:55.931471 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7317d-bd78-410c-a66a-dd45f4437690 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:55.931471 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7317d-bd78-410c-a66a-dd45f4437690 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:55.931645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7317d-bd78-410c-a66a-dd45f4437690 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:55.933081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 32/65] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 05:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 05:59:55.933731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7317d-bd78-410c-a66a-dd45f4437690 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:55.962468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.021612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a244f-c9e8-400a-9578-268164e17d0c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.024452 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.042045 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7317d-bd78-410c-a66a-dd45f4437690 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.058905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.060491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.060780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.061054 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.072160 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:59:56.076241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.076853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:59:56.077005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:56.078163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 05:59:56.079563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 05:59:56.080227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.080227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.080370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 34/66] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 05:59:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:59:56.080440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:56.080440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.080582 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.080665 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2870f-872f-404b-81ff-5dbf964c4cb2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.092582 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 33/67] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:59:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.096669 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.097022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.097315 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.115053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 05:59:56.116299 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 05:59:56.181857 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 34/68] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 05:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 05:59:56.182608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.188085 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08019-d4c0-482f-bb14-55b9e7654e95 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.202342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.224986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.225672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.225962 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.226242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.226919 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:56.227232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.228744 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.237288 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.237907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.269097 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.301361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd37e-b582-4311-bd84-1c040fe25d85 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.339336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.339969 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.340257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.340536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] PATH_INFO: `/v3/roles/member` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.343492 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252d9e-00eb-4e25-bc64-27eebe6f7144 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.352124 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 05:59:56.357427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 05:59:56.358556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 05:59:56.360887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.386739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.387375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.387659 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.387933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.388593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:56.388896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.390271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.402990 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 36/71] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.402990 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.454108 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.477530 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f06cf4-7236-4ad5-9894-9cb5f04d269e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.496802 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.509777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882abf2-a8bb-46e6-bf4a-3ee543f27b94 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.513229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.513870 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.514151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.514425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] PATH_INFO: `/v3/users/c4cc297532144f6f8f4534b8faf80b42` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.527506 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] RBAC: Authorizing `identity:get_user(user_id=c4cc297532144f6f8f4534b8faf80b42)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.528985 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.530576 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/users/c4cc297532144f6f8f4534b8faf80b42 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.531394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.543858 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.544502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.544787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.545067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] PATH_INFO: `/v3/users/bf714812a1a846729a9dd3749da0e163` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.546213 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] RBAC: Authorizing `identity:get_user(user_id=bf714812a1a846729a9dd3749da0e163)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.548265 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.549837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 37/73] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/users/bf714812a1a846729a9dd3749da0e163 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.554307 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.639362 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544363b1-cda8-4d02-b4d7-405d949b9b9c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.652636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4913c93-7e29-455b-a2f7-aea1452ec99c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.673386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.673997 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.674284 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.674562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.675237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 05:59:56.675575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.676977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.679182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.686054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 37/74] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.686855 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.691916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.692559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.692840 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.693113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] PATH_INFO: `/v3/users/c4cc297532144f6f8f4534b8faf80b42` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.694287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] RBAC: Authorizing `identity:get_user(user_id=c4cc297532144f6f8f4534b8faf80b42)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.695769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.697339 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 38/75] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/users/c4cc297532144f6f8f4534b8faf80b42 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.698093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.723334 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.790902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286b219-fd46-456f-8675-38a8697b735d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.797073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb356eaa-bc97-4fb9-a17f-7b634c95f67d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.823851 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.824483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.824762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.825035 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.826147 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.827623 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.829239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 38/76] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.830003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.832829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.833475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.834023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.834327 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.835599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.837031 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.843290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 39/77] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.843290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.895101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.928399 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:56.931147 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e9e1-7da8-4df6-a975-733b4902d243 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.937950 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394ba1f-d9e4-422b-bdcc-d0c198e22082 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:56.964372 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.964998 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.965281 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.965558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.966682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.968175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.970307 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 39/78] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.972076 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:56.972220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:56.972717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:56.973005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:56.973286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] PATH_INFO: `/v3/users/c4cc297532144f6f8f4534b8faf80b42` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:56.974551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] RBAC: Authorizing `identity:get_user(user_id=c4cc297532144f6f8f4534b8faf80b42)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:56.976000 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:56.981881 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 40/79] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/users/c4cc297532144f6f8f4534b8faf80b42 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:56.981881 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:57.070577 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1485-752a-48bc-b538-0f9c3d4a58f7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.080695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc527b-3bbe-45c7-b617-9c92a87eff27 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.104302 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.104931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.105216 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.105501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.106626 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.108123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.109769 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 40/80] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:59:56 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:57.110511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:57.116816 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:57.117754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.118235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.118519 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.118794 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.119497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.120935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.210510 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 41/81] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 05:59:56 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 05:59:57.210510 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:57.220782 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:57.223373 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f43676-1e23-496b-a1a7-4092979700a0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.260483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.261089 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.261378 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.261660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23/users/c4cc297532144f6f8f4534b8faf80b42/roles/abebdef18b5648ea9d768c2b83eb2ddd` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.263905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] RBAC: Authorizing `identity:create_grant(project_id=8d062649850c4cc280349e14d6730b23, user_id=c4cc297532144f6f8f4534b8faf80b42, role_id=abebdef18b5648ea9d768c2b83eb2ddd)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.265216 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.314023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394fa7-62b4-4c6e-98c7-b9da41186006 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.347010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 41/82] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:59:57 2022] PUT /identity/v3/projects/8d062649850c4cc280349e14d6730b23/users/c4cc297532144f6f8f4534b8faf80b42/roles/abebdef18b5648ea9d768c2b83eb2ddd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:59:57.348387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:57.365769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.366982 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.367463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.368030 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23/users/bf714812a1a846729a9dd3749da0e163/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.370240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d062649850c4cc280349e14d6730b23, user_id=bf714812a1a846729a9dd3749da0e163, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.372608 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.427009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:57.475202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 42/83] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:59:57 2022] PUT /identity/v3/projects/8d062649850c4cc280349e14d6730b23/users/bf714812a1a846729a9dd3749da0e163/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:59:57.475913 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b2b1-1e9e-4e2a-862a-b4d1c417548c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.476190 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:57.530446 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.531027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.531926 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.532263 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] PATH_INFO: `/v3/projects/6477c86a969e454d98e551e231c4284c` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.533703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] RBAC: Authorizing `identity:get_project(project_id=6477c86a969e454d98e551e231c4284c)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.535461 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.537392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 42/84] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 05:59:57 2022] GET /identity/v3/projects/6477c86a969e454d98e551e231c4284c => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 05:59:57.543454 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 05:59:57.592426 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:57.627484 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1974673-335f-46ad-94fc-196b69a03fa4 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.674257 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453f74d-d4f9-424e-a038-2fc9d550aeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.674953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453f74d-d4f9-424e-a038-2fc9d550aeef None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.675241 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453f74d-d4f9-424e-a038-2fc9d550aeef None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.675552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453f74d-d4f9-424e-a038-2fc9d550aeef None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23/users/c4cc297532144f6f8f4534b8faf80b42/roles/e5f3830b373b402693826265019b9344` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.678014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453f74d-d4f9-424e-a038-2fc9d550aeef None admin] RBAC: Authorizing `identity:create_grant(project_id=8d062649850c4cc280349e14d6730b23, user_id=c4cc297532144f6f8f4534b8faf80b42, role_id=e5f3830b373b402693826265019b9344)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.681474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453f74d-d4f9-424e-a038-2fc9d550aeef None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.683645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd47e8-278b-4695-8695-59c3ae9df3c5 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 05:59:57.725113 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 05:59:57.732491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95de7da-d47b-43d1-bdbd-ed48c307f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 05:59:57.733268 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95de7da-d47b-43d1-bdbd-ed48c307f810 None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 05:59:57.733650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95de7da-d47b-43d1-bdbd-ed48c307f810 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 05:59:57.734024 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95de7da-d47b-43d1-bdbd-ed48c307f810 None admin] PATH_INFO: `/v3/projects/6477c86a969e454d98e551e231c4284c/users/c4cc297532144f6f8f4534b8faf80b42/roles/abebdef18b5648ea9d768c2b83eb2ddd` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 05:59:57.736853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95de7da-d47b-43d1-bdbd-ed48c307f810 None admin] RBAC: Authorizing `identity:create_grant(project_id=6477c86a969e454d98e551e231c4284c, user_id=c4cc297532144f6f8f4534b8faf80b42, role_id=abebdef18b5648ea9d768c2b83eb2ddd)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 05:59:57.738326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95de7da-d47b-43d1-bdbd-ed48c307f810 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 05:59:57.772302 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 43/85] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:59:57 2022] PUT /identity/v3/projects/8d062649850c4cc280349e14d6730b23/users/c4cc297532144f6f8f4534b8faf80b42/roles/e5f3830b373b402693826265019b9344 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 05:59:57.826884 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 43/86] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 05:59:57 2022] PUT /identity/v3/projects/6477c86a969e454d98e551e231c4284c/users/c4cc297532144f6f8f4534b8faf80b42/roles/abebdef18b5648ea9d768c2b83eb2ddd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:00:00.037480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3ede7-5e9d-46aa-af90-ae52e11dddcb None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.038044 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3ede7-5e9d-46aa-af90-ae52e11dddcb None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.038044 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3ede7-5e9d-46aa-af90-ae52e11dddcb None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.039396 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 44/87] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:00.045062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d936ad-bb7d-445e-a9e1-5a7c36de42fc None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.045475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d936ad-bb7d-445e-a9e1-5a7c36de42fc None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.045681 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d936ad-bb7d-445e-a9e1-5a7c36de42fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.071737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-33d936ad-bb7d-445e-a9e1-5a7c36de42fc None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:00.072946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-33d936ad-bb7d-445e-a9e1-5a7c36de42fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:00.160521 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 44/88] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:00.720634 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3ede7-5e9d-46aa-af90-ae52e11dddcb None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:00.759590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9beef8-98db-444f-99a7-7682b2a17b87 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.760013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9beef8-98db-444f-99a7-7682b2a17b87 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.760294 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9beef8-98db-444f-99a7-7682b2a17b87 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.761657 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 45/89] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:00.767226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc66b-ab2b-4ced-a5e9-0f826e5aec9b None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.767545 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc66b-ab2b-4ced-a5e9-0f826e5aec9b None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.767825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc66b-ab2b-4ced-a5e9-0f826e5aec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.794697 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-184dc66b-ab2b-4ced-a5e9-0f826e5aec9b None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:00.795941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-184dc66b-ab2b-4ced-a5e9-0f826e5aec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:00.871220 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3ede7-5e9d-46aa-af90-ae52e11dddcb None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:00.912371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403aef9-80b9-437d-8577-dcfe21058e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:00.913016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403aef9-80b9-437d-8577-dcfe21058e31 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.913299 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403aef9-80b9-437d-8577-dcfe21058e31 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.913573 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403aef9-80b9-437d-8577-dcfe21058e31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.913693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 46/90] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:00.915133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97555d95-4526-4eb5-9007-9399588c504f None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.915475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97555d95-4526-4eb5-9007-9399588c504f None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.915998 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97555d95-4526-4eb5-9007-9399588c504f None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.917023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:00.918379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 47/91] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:00.919443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04402bc-7b89-44c3-93de-26b9694e168a None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.919723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04402bc-7b89-44c3-93de-26b9694e168a None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.919993 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04402bc-7b89-44c3-93de-26b9694e168a None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.921238 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 48/92] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:00.927637 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.930496 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.930496 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.932155 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403aef9-80b9-437d-8577-dcfe21058e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-5403aef9-80b9-437d-8577-dcfe21058e31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:00.938326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:00.939324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:00.940208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 45/93] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:00.944723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0b93-3541-4184-9ef4-ae8e3afb2706 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:00.945008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0b93-3541-4184-9ef4-ae8e3afb2706 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:00.945283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0b93-3541-4184-9ef4-ae8e3afb2706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:00.948612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:00.950195 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:00.965636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-93ad0b93-3541-4184-9ef4-ae8e3afb2706 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:00.966935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-93ad0b93-3541-4184-9ef4-ae8e3afb2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:01.010388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.031682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 49/94] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:01.039375 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.059901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 46/95] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:01.061571 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4766b-f9e8-4d97-a185-dd269f5b320f None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.061863 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4766b-f9e8-4d97-a185-dd269f5b320f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.062149 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4766b-f9e8-4d97-a185-dd269f5b320f None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.064099 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 47/96] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:01.070267 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.070691 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.071098 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.099792 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:01.101002 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:01.271754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 48/97] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:01.273946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a6ecc-b26c-4fce-9914-8e7b2448996c None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.280320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.344360 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.344981 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.345265 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.345544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.346333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:01.346647 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:01.348175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffda92-c1fc-4df8-99cc-551fa4ea0eb1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:01.357107 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 50/98] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:01.358317 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ca23b-2101-4b8d-849c-06e7032ada88 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.358609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ca23b-2101-4b8d-849c-06e7032ada88 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.358890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ca23b-2101-4b8d-849c-06e7032ada88 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.363330 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 51/99] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:01.365155 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ca23b-2101-4b8d-849c-06e7032ada88 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.407667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.427365 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.453999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2678c8-0dcf-4ed4-bada-8d8fe3203330 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.491161 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ca23b-2101-4b8d-849c-06e7032ada88 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.495149 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398ecfc-3c65-4511-b72b-64028a66257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.495823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398ecfc-3c65-4511-b72b-64028a66257d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.496102 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398ecfc-3c65-4511-b72b-64028a66257d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.496375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398ecfc-3c65-4511-b72b-64028a66257d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.507242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398ecfc-3c65-4511-b72b-64028a66257d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-7398ecfc-3c65-4511-b72b-64028a66257d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:01.512103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:01.513063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:01.513916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 49/100] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:01.515705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.515993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.516262 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.532034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.532595 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.532881 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.533225 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] PATH_INFO: `/v3/users/c4cc297532144f6f8f4534b8faf80b42` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.533982 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:01.534295 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] RBAC: Authorizing `identity:get_user(user_id=c4cc297532144f6f8f4534b8faf80b42)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:01.535169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:01.535773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:01.537527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 52/101] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/users/c4cc297532144f6f8f4534b8faf80b42 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:01.540229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.604215 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 50/102] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:01.608400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.652717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05789926-2754-4d16-80bf-4fc91ba5aea7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.653984 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.663560 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.692428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.693327 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.693615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.694041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.705899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:01.710931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:01.712081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:01.713096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:01.713777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaef1bb-0a5b-4a26-936d-22eff86aaf3b None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.719364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 53/103] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:01.719364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.750719 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.751450 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.751733 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.752024 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.762549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:01.768084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:01.769014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:01.769864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 51/104] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:01.770566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.835689 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f5a57-4bdd-44dd-ae01-52c8dee85ab3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.871194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.871845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.872128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.872231 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1eab3-e64b-4aae-90a9-3cf87d146b1a None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:01.872415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.873102 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:01.873395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:01.875023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:01.879261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.883587 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 54/105] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:01.884390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:01.910222 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:01.911167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:01.911721 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:01.912070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:01.916664 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:01.979045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:01.980537 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:01.985521 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a6d55-07ca-4b8a-b2e7-0b10ce903f50 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.018213 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.018842 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.019126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.019434 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.029580 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:02.034947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:02.036021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 55/106] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:02.036904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.081011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 52/107] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:02.081683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.089214 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.128157 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.136783 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ce728-1011-447f-bae8-61644fe76f33 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.171394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.172101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.172389 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.172668 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.173342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:02.173948 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.175062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.179216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6df305-b1bd-4341-9bda-dffe45897a20 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.183193 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 56/108] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.183193 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.214364 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.214982 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.215270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.215577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.216242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:02.216552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.217934 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.226528 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 53/109] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.227133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.286277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643e92b-15c1-4f2a-b1dd-7cc326b1cb49 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.293987 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.319935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.320572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.320861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.321142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.322272 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.323772 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.325340 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 57/110] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.327719 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84fe77-0870-43f8-803c-3c14334722ea None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.328688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.353427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.364014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.365822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.365822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.365822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] PATH_INFO: `/v3/users/41c0bfd2f9e14a23a0dce6871183bfc5` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.380508 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] RBAC: Authorizing `identity:get_user(user_id=41c0bfd2f9e14a23a0dce6871183bfc5)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.381969 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.388074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 54/111] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/users/41c0bfd2f9e14a23a0dce6871183bfc5 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.388074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.428469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b44388-f4a6-43d0-9635-71d97cfa83ea None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.463818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.464013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.464240 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.464525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.465211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:02.465524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.466921 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.475426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 58/112] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.477713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.489655 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9f956-433a-4288-abc2-575503e4246e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.515777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.524492 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.529460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.529460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.529460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.529460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.529460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.530011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 55/113] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.530011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.568507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.583487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f7b92-07d6-436a-9012-f6c15ddc902c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.616731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.617300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.617590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.617867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] PATH_INFO: `/v3/users/c4cc297532144f6f8f4534b8faf80b42` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.619007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] RBAC: Authorizing `identity:get_user(user_id=c4cc297532144f6f8f4534b8faf80b42)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.620536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.622126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 59/114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/users/c4cc297532144f6f8f4534b8faf80b42 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.622126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.634885 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b402981-5638-4e5d-a6c7-bed60876fef8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.669450 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.670088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.670374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.670653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] PATH_INFO: `/v3/users/bf714812a1a846729a9dd3749da0e163` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.672409 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] RBAC: Authorizing `identity:get_user(user_id=bf714812a1a846729a9dd3749da0e163)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.673843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.675475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 56/115] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/users/bf714812a1a846729a9dd3749da0e163 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.676240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.726375 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62d82-59db-45c7-a8df-aab73df5045d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.737105 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.758941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.759585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.759874 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.760152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.760805 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.762243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.775027 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38a16a-f2d8-41af-a936-cda4f480ee8d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.810192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.810823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.811160 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.811509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.812640 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.814112 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.815779 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 57/116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.816560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.844743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 60/117] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:00:02 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:02.845437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:02.848925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.942383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298d7eb-b264-42bc-b6b8-1260566e69ba None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.960346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8e21-1cb1-4e9e-a1e9-071a12f6d6c7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:02.976146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:02.982432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:02.983085 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:02.983401 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:02.984180 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] PATH_INFO: `/v3/users/c4cc297532144f6f8f4534b8faf80b42` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:02.986311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] RBAC: Authorizing `identity:get_user(user_id=c4cc297532144f6f8f4534b8faf80b42)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:02.987874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:02.989585 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 58/118] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/users/c4cc297532144f6f8f4534b8faf80b42 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:02.990376 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.006352 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.006997 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.007286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.007596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.008360 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] RBAC: Adding query filter params (role.id=abebdef18b5648ea9d768c2b83eb2ddd, scope.project.id=8d062649850c4cc280349e14d6730b23, user.id=c4cc297532144f6f8f4534b8faf80b42) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:03.008677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.010165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.027618 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 61/119] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/role_assignments?user.id=c4cc297532144f6f8f4534b8faf80b42&scope.project.id=8d062649850c4cc280349e14d6730b23&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:03.027618 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.118799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.143004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1565b-8e95-4bea-8f44-08af80d0c112 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.156730 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2be97b-ff76-4b2d-8881-858f2cb07c7d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.195174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.195761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.196051 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.196333 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.200838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.201174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.201467 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.201746 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.284254 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.285233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.287135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.288773 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.290202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.369017 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.410011 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 62/120] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:03.410011 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.410295 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 59/121] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:03.412907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.513747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb453d-df98-4caf-b9ea-060ac173beed None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.526783 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.531600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7044d4-3ccb-49ac-955f-ce77fa8376ca None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.553152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.553822 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.554107 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.554383 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23/users/41c0bfd2f9e14a23a0dce6871183bfc5/roles/75b9861121374bc2a5fcb0cee200c294` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.556655 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] RBAC: Authorizing `identity:create_grant(project_id=8d062649850c4cc280349e14d6730b23, user_id=41c0bfd2f9e14a23a0dce6871183bfc5, role_id=75b9861121374bc2a5fcb0cee200c294)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.558111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.567487 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.568109 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.568393 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.568671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.639344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 63/122] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:03 2022] PUT /identity/v3/projects/8d062649850c4cc280349e14d6730b23/users/41c0bfd2f9e14a23a0dce6871183bfc5/roles/75b9861121374bc2a5fcb0cee200c294 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:03.639344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.644878 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.646388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.672305 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.768521 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b9f38-0949-42ce-8631-a85631e4f84a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.775888 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.780858 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 60/123] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:03.786900 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.819007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.820174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.820613 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.821033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.822537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.824475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.827394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 64/124] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:03.832508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.921287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859df4c0-1a9d-45af-b0f4-8102e23919a3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.943224 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc1522-3bc9-493c-970e-f6d6cb2fd38f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:03.964943 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:03.969050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.969775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.970127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.970471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.971712 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.973336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.975288 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 61/125] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:03.976135 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:03.983350 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:03.983558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:03.983922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:03.986043 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] PATH_INFO: `/v3/projects/6477c86a969e454d98e551e231c4284c` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:03.987764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] RBAC: Authorizing `identity:get_project(project_id=6477c86a969e454d98e551e231c4284c)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:03.989528 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:03.992140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 65/126] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/projects/6477c86a969e454d98e551e231c4284c => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:03.993874 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:04.058594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:04.096144 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2e807-b8fa-45a8-b5fa-50c1775ea91a None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:04.121429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce132115-831c-42af-9d96-ca421fb7568f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:04.141037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:04.141666 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:04.141953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:04.142232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:04.142979 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=8d062649850c4cc280349e14d6730b23, user.id=bf714812a1a846729a9dd3749da0e163) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:04.143292 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:04.145028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:04.151898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 62/127] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/role_assignments?user.id=bf714812a1a846729a9dd3749da0e163&scope.project.id=8d062649850c4cc280349e14d6730b23&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:04.152716 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:04.162197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:04.163411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:04.163917 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:04.164397 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:04.165527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] RBAC: Adding query filter params (role.id=e5f3830b373b402693826265019b9344, scope.project.id=8d062649850c4cc280349e14d6730b23, user.id=c4cc297532144f6f8f4534b8faf80b42) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:04.166094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:04.167930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a1c53-f519-477d-a08b-c2b3a0477734 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:04.183253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 66/128] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:03 2022] GET /identity/v3/role_assignments?user.id=c4cc297532144f6f8f4534b8faf80b42&scope.project.id=8d062649850c4cc280349e14d6730b23&role.id=e5f3830b373b402693826265019b9344 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:04.270498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0501df-5bd3-42a4-aee2-299449d89525 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:04.304591 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:04.322358 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:04.322358 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:04.322358 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:04.322358 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:04.322875 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] RBAC: Adding query filter params (role.id=abebdef18b5648ea9d768c2b83eb2ddd, scope.project.id=6477c86a969e454d98e551e231c4284c, user.id=c4cc297532144f6f8f4534b8faf80b42) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:04.322875 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:04.323986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15d851-1081-4a2e-838d-e18b72b5b358 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:04.330942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 63/129] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:04 2022] GET /identity/v3/role_assignments?user.id=c4cc297532144f6f8f4534b8faf80b42&scope.project.id=6477c86a969e454d98e551e231c4284c&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:05.143517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb0834-8a73-49bf-84bb-b79d45fc7cf6 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:05.144165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb0834-8a73-49bf-84bb-b79d45fc7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:05.144165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb0834-8a73-49bf-84bb-b79d45fc7cf6 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:05.145416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 67/130] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:05.150799 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:05.151241 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:05.151510 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:05.169965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:05.171139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:05.235178 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 64/131] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:05.502059 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb0834-8a73-49bf-84bb-b79d45fc7cf6 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:05.530535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:05.604653 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb0834-8a73-49bf-84bb-b79d45fc7cf6 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:05.638002 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda980-1db2-4143-9963-855e95386246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:05.638627 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda980-1db2-4143-9963-855e95386246 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:05.638909 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda980-1db2-4143-9963-855e95386246 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:05.639181 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda980-1db2-4143-9963-855e95386246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:05.640872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda980-1db2-4143-9963-855e95386246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:05.642324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda980-1db2-4143-9963-855e95386246 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:05.643901 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 68/132] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:05.649344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:05.749356 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69a9d-1705-41fb-b9c9-61b0a2531634 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:05.768184 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:05.778256 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca88cb-0246-42b1-b901-8b6e87250c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:05.778872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca88cb-0246-42b1-b901-8b6e87250c0b None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:05.779154 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca88cb-0246-42b1-b901-8b6e87250c0b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:05.779465 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca88cb-0246-42b1-b901-8b6e87250c0b None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:05.780140 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca88cb-0246-42b1-b901-8b6e87250c0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:05.782102 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca88cb-0246-42b1-b901-8b6e87250c0b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:05.867020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 65/133] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:05 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:05.976967 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf5396-2821-4c2d-bdde-915fbe864c0a None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:05.977279 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf5396-2821-4c2d-bdde-915fbe864c0a None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:05.977558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf5396-2821-4c2d-bdde-915fbe864c0a None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:05.978926 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 69/134] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:05.984898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:05.985272 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:05.985615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:06.005129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:06.006261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:06.075395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 66/135] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:06.341930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf5396-2821-4c2d-bdde-915fbe864c0a None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:06.442482 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:06.450066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf5396-2821-4c2d-bdde-915fbe864c0a None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:06.487020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:06.487695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:06.487976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:06.488252 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:06.499104 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:06.504625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:06.505630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:06.506524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 70/136] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:06.513551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:06.615066 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f8630-d830-42fa-a928-0dcc6e87a75f None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:06.626157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:06.650187 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:06.651134 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:06.651628 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:06.652152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:06.653087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:06.653599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:06.655166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:06.664794 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 67/137] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:06.673083 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:06.778738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce8765-a747-463a-b5b4-61278953231e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:06.814051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:06.814688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:06.814970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:06.815246 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] PATH_INFO: `/v3/users/41c0bfd2f9e14a23a0dce6871183bfc5` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:06.816573 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] RBAC: Authorizing `identity:get_user(user_id=41c0bfd2f9e14a23a0dce6871183bfc5)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:06.818045 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:06.820020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 71/138] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:06 2022] GET /identity/v3/users/41c0bfd2f9e14a23a0dce6871183bfc5 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:06.825704 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:06.938872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fc62a-4442-4d96-a6a5-7d71cea1c6c7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:06.973464 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:06.974085 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:06.974424 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:06.974700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:07.019644 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:07.049725 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:07.051170 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:07.157700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 68/139] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:07.163646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:07.195758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:07.269770 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db885e-e433-47b2-b106-968ee18abe03 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:07.304996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d4e8-e5b8-4ea0-9071-8773b0b0dcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:07.306268 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d4e8-e5b8-4ea0-9071-8773b0b0dcc8 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:07.306775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d4e8-e5b8-4ea0-9071-8773b0b0dcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:07.307333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d4e8-e5b8-4ea0-9071-8773b0b0dcc8 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:07.308775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d4e8-e5b8-4ea0-9071-8773b0b0dcc8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:07.310484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d4e8-e5b8-4ea0-9071-8773b0b0dcc8 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:07.313161 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 72/140] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:07 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:07.323322 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:07.393142 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:07.424058 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5517193-7b6d-49c5-bcf9-38c0ae1018d9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:07.470694 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:07.471245 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:07.471565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:07.471842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:07.472604 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] RBAC: Adding query filter params (role.id=75b9861121374bc2a5fcb0cee200c294, scope.project.id=8d062649850c4cc280349e14d6730b23, user.id=41c0bfd2f9e14a23a0dce6871183bfc5) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:07.472919 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:07.474427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57290b72-f142-49e1-86d9-bc3b95b22851 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:07.484866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 69/141] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:07 2022] GET /identity/v3/role_assignments?user.id=41c0bfd2f9e14a23a0dce6871183bfc5&scope.project.id=8d062649850c4cc280349e14d6730b23&role.id=75b9861121374bc2a5fcb0cee200c294 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:08.155049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21917f-8dba-4f23-b256-8214bdced1c0 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.155629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21917f-8dba-4f23-b256-8214bdced1c0 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.155629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21917f-8dba-4f23-b256-8214bdced1c0 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.157672 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 73/142] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:08.162562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974fe87-d75f-40ac-bedc-26d263697545 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.162952 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974fe87-d75f-40ac-bedc-26d263697545 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.163110 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974fe87-d75f-40ac-bedc-26d263697545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.181733 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-3974fe87-d75f-40ac-bedc-26d263697545 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:08.183142 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-3974fe87-d75f-40ac-bedc-26d263697545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:08.256164 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 70/143] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:08.326664 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c7d79-df49-4343-890e-512103933d4d None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.326945 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c7d79-df49-4343-890e-512103933d4d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.327453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c7d79-df49-4343-890e-512103933d4d None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.329318 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 74/144] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:08.334541 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.334836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.335116 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.353959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:08.355150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:08.399498 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0041-5239-4929-a116-d3ca3d7cb7c3 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.399785 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0041-5239-4929-a116-d3ca3d7cb7c3 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.400262 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0041-5239-4929-a116-d3ca3d7cb7c3 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.401584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 75/145] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:08.406629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.407122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.407491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.417812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:08.425757 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 71/146] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:08.431475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:08.431475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:08.503346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 76/147] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:08.524039 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:08.624353 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221fd03-9d75-45d7-91c9-f50a473ebb25 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:08.658317 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:08.658897 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.659181 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.659486 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.669429 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:08.674903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:08.676015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:08.676907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 72/148] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:08.682061 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:08.701659 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:08.709406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:08.762514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:08.797133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c76161-1221-4c61-a8f2-5b4102765b70 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:08.807631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88a0df-4123-46cc-b35c-48d1914015c3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:08.831820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:08.832434 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.832712 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.832986 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.833659 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:08.833959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:08.835384 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:08.841493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:08.842121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.842400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.842670 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.843460 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 77/149] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:08.844039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:08.857178 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:08.862540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:08.863589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 73/150] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:00:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:08.864463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:08.934883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:08.950357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a15fd9-d1e6-4363-bb91-27bfd45896d6 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:08.965627 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e624-a195-43c7-8df0-1bcf3b86efb6 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:08.981789 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:08.984251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:08.984883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:08.985168 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:08.985440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:08.986569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:08.988061 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:08.989776 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 78/151] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:08.990376 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.003915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.004552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.004828 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.005098 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] PATH_INFO: `/v3/users/376bceb5e33247ec812e693cb2f2e4c7` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.018568 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] RBAC: Authorizing `identity:get_user(user_id=376bceb5e33247ec812e693cb2f2e4c7)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.020055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.021814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 74/152] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:08 2022] GET /identity/v3/users/376bceb5e33247ec812e693cb2f2e4c7 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:09.022379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.092478 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186373-77d0-402e-82e2-826fccfdcbeb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.118466 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f1f76-dbf0-43f8-a767-58d1ad6208ed None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.126233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.126835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.127112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.127423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.128088 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:09.128390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.129752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.137630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 79/153] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:09.138741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.153173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.153791 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.153931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:09.154070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.154334 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.154991 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.156447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.235187 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d71564-2bb1-452f-9292-db6ef0396591 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.268094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.268719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.268999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.269275 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.270397 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.271882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.273507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 80/154] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:09 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:09.274264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.277172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 75/155] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:09 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:09.292646 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.331536 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:09.370651 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:09.390798 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a96724-0def-4ddb-9a21-63f58104f8a1 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.416783 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f842e1b-366f-4d47-a89c-c6badbcd3816 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.427893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.428545 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.428831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.429116 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] PATH_INFO: `/v3/users/376bceb5e33247ec812e693cb2f2e4c7` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.430300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] RBAC: Authorizing `identity:get_user(user_id=376bceb5e33247ec812e693cb2f2e4c7)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.432164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.433812 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 81/156] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:09 2022] GET /identity/v3/users/376bceb5e33247ec812e693cb2f2e4c7 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:09.440076 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.463419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.464103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.465737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.465737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/376bceb5e33247ec812e693cb2f2e4c7/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.469185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddd1a08322a42ffb5981fd594ba15aa, user_id=376bceb5e33247ec812e693cb2f2e4c7, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.470641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.552931 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 76/157] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:09 2022] PUT /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/376bceb5e33247ec812e693cb2f2e4c7/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:00:09.557798 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643d07-b922-4ac2-9fd5-241daaed57d7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.601508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82864d-05b2-49e6-8c5f-d7a9228e852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.602239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82864d-05b2-49e6-8c5f-d7a9228e852f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.602571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82864d-05b2-49e6-8c5f-d7a9228e852f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.602850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82864d-05b2-49e6-8c5f-d7a9228e852f None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.604041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82864d-05b2-49e6-8c5f-d7a9228e852f None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.605541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82864d-05b2-49e6-8c5f-d7a9228e852f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.607280 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 82/158] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:09 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:09.613536 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:09.718803 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:09.728904 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165be98-6856-4972-85c3-f5ff0c4b0734 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:09.769681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629915ba-f879-414a-b032-0514e2b47f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:09.770360 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629915ba-f879-414a-b032-0514e2b47f63 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:09.770646 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629915ba-f879-414a-b032-0514e2b47f63 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:09.770914 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629915ba-f879-414a-b032-0514e2b47f63 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/376bceb5e33247ec812e693cb2f2e4c7/roles/e5f3830b373b402693826265019b9344` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:09.773137 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629915ba-f879-414a-b032-0514e2b47f63 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddd1a08322a42ffb5981fd594ba15aa, user_id=376bceb5e33247ec812e693cb2f2e4c7, role_id=e5f3830b373b402693826265019b9344)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:09.774594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629915ba-f879-414a-b032-0514e2b47f63 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:09.837802 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 77/159] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:09 2022] PUT /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/376bceb5e33247ec812e693cb2f2e4c7/roles/e5f3830b373b402693826265019b9344 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:11.561787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e378a-148f-47de-aa4f-c305a8e13faa None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:11.562886 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e378a-148f-47de-aa4f-c305a8e13faa None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:11.562989 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e378a-148f-47de-aa4f-c305a8e13faa None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:11.564582 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 83/160] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:11.569974 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c31b9a-56ca-4ac0-9d51-2b36f151d91e None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:11.570497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c31b9a-56ca-4ac0-9d51-2b36f151d91e None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:11.570497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c31b9a-56ca-4ac0-9d51-2b36f151d91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:11.590906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-96c31b9a-56ca-4ac0-9d51-2b36f151d91e None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:11.592204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-96c31b9a-56ca-4ac0-9d51-2b36f151d91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:11.669597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 78/161] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:11.703912 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2761f1a-52b1-46ca-8663-3aaf54f1826c None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:11.704186 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2761f1a-52b1-46ca-8663-3aaf54f1826c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:11.704468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2761f1a-52b1-46ca-8663-3aaf54f1826c None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:11.706068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 84/162] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:11.711410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:11.711703 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:11.711981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:11.730888 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:11.732042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:11.793066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c4fbb-ee95-481e-a0db-daf8713a07b8 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:11.793721 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c4fbb-ee95-481e-a0db-daf8713a07b8 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:11.794214 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c4fbb-ee95-481e-a0db-daf8713a07b8 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:11.795830 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 85/163] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:11.801340 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:11.801866 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:11.802481 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:11.802598 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 79/164] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:11.827007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:11.828297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:11.841914 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:11.896902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 86/165] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:11.942318 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:11.955314 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.046951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9eba5-9a8f-4d63-a2be-91847f911d75 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.080684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.081662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890af96-353f-4789-83aa-1b5b825e2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.082313 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890af96-353f-4789-83aa-1b5b825e2565 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.082588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890af96-353f-4789-83aa-1b5b825e2565 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.082861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890af96-353f-4789-83aa-1b5b825e2565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.094219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890af96-353f-4789-83aa-1b5b825e2565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-b890af96-353f-4789-83aa-1b5b825e2565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:12.099554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:12.100701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:12.101753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 80/166] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:12.102617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375b3a7-ab38-4ee0-aa6b-19fd9be35987 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.102617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375b3a7-ab38-4ee0-aa6b-19fd9be35987 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.102617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375b3a7-ab38-4ee0-aa6b-19fd9be35987 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.103953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 81/167] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:12.106346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375b3a7-ab38-4ee0-aa6b-19fd9be35987 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.188254 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.190099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb243c94-ebd8-4358-9359-f494361bb272 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.220311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375b3a7-ab38-4ee0-aa6b-19fd9be35987 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.224391 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a97b62-3567-4506-9555-0acd00148646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.225062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97b62-3567-4506-9555-0acd00148646 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.225341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97b62-3567-4506-9555-0acd00148646 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.225610 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97b62-3567-4506-9555-0acd00148646 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.226706 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a97b62-3567-4506-9555-0acd00148646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.228190 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a97b62-3567-4506-9555-0acd00148646 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.229723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 87/168] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:12.230818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.231095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.231392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.249009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:12.250211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:12.258028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.258664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.258938 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.259206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.259889 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:12.260192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.261403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.261551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.270020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 82/169] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:12.271926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.319381 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 88/170] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:12.319381 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.379212 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f7fd-b13b-4808-97eb-d3558d759cb2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.410172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.415263 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.415914 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.416207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.416459 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.425185 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3337c6-63be-4212-82b8-cb40e1ee6378 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.428874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:12.434014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:12.434959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:12.435936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 83/171] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:12.436723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.472455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.473082 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.473367 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.473641 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.474294 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.475786 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.545106 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcab726-afd4-4dc0-88e1-a39206b78629 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.578556 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.579175 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.579449 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.579683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] PATH_INFO: `/v3/users/f58c5f4aee044c908131c3e9928a8a2d` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.580109 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 89/172] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:12 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:12.581293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.584008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.608130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] RBAC: Authorizing `identity:get_user(user_id=f58c5f4aee044c908131c3e9928a8a2d)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.615905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.615905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 84/173] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/users/f58c5f4aee044c908131c3e9928a8a2d => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:12.615905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.670643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.709724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8667aec-f82a-4197-b843-3457732014b0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.729120 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5fe4-54a0-4202-add2-3fd62261797e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.750051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.750670 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.750952 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.751473 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.751938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:12.752241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.753640 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.764653 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 90/174] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:12.766193 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.773275 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.773922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.774197 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.774471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.788369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:12.793821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:12.794787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:12.795696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 85/175] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:12.796937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.844142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.899959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72463658-e9b2-4cc3-9b08-c3cc29b76f44 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.922386 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20086fc7-bbae-4443-bfda-1142826d70c9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:12.936101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:12.942702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.943371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.943654 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.943922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.945074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.946598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.948774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 91/176] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:12.949528 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:12.977870 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:12.979249 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:12.979585 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:12.980128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] PATH_INFO: `/v3/users/376bceb5e33247ec812e693cb2f2e4c7` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:12.981677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=376bceb5e33247ec812e693cb2f2e4c7)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:12.984416 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:12.987286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 86/177] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/users/376bceb5e33247ec812e693cb2f2e4c7 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:12.988229 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.057796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeb490-4bb0-4284-b5b3-15be325e001e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:13.092321 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:13.097831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:13.098368 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:13.098647 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:13.098922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:13.099697 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:13.109245 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:13.109245 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:13.110673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 92/178] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:13.116166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.125181 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36153-1663-47c9-bf9e-b7761e7b1fa1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:13.174055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:13.175336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:13.175784 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:13.176476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/f58c5f4aee044c908131c3e9928a8a2d/roles/abebdef18b5648ea9d768c2b83eb2ddd` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:13.179590 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] RBAC: Authorizing `identity:create_grant(project_id=eddd1a08322a42ffb5981fd594ba15aa, user_id=f58c5f4aee044c908131c3e9928a8a2d, role_id=abebdef18b5648ea9d768c2b83eb2ddd)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:13.182531 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:13.225534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa805-22ea-4040-9475-b02d4c91c75c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:13.261024 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 87/179] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:12 2022] PUT /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/f58c5f4aee044c908131c3e9928a8a2d/roles/abebdef18b5648ea9d768c2b83eb2ddd => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:13.262953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.263943 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:13.267480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:13.267480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:13.267480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:13.320315 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:13.396642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1b96-c228-479c-9588-5c9698080e9f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:13.421192 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:13.439945 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:13.441669 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:13.442238 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:13.442730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] PATH_INFO: `/v3/users/376bceb5e33247ec812e693cb2f2e4c7` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:13.444338 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] RBAC: Authorizing `identity:get_user(user_id=376bceb5e33247ec812e693cb2f2e4c7)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:13.446331 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:13.448594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 88/180] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:13 2022] GET /identity/v3/users/376bceb5e33247ec812e693cb2f2e4c7 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:13.450457 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:13.455396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:13.456582 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.596392 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244133d-3e71-4ff5-afd4-ed36d5bbdf6b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:13.601620 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 93/181] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:13.606457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:13.607819 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.653660 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:13.654215 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:13.654488 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:13.654759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:13.712425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:13.719480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f115a2e-1df6-4215-a1b6-a76fc9d97186 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:13.775451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:13.776080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:13.776361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:13.776633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:13.778427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:13.780541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:13.784698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 94/182] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:13 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:13.788115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:13.789685 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:13.792337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.980972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 89/183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:13.986594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:13.998177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1e2b8-112e-43e3-ba46-07e79c483b65 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:14.000090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:14.078499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:14.079126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:14.079429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:14.079704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:14.080509 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=eddd1a08322a42ffb5981fd594ba15aa, user.id=376bceb5e33247ec812e693cb2f2e4c7) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:14.080819 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:14.082273 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:14.098206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab1327-01f1-4fcd-92c6-822fd5d7a0ad None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:14.102246 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 95/184] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:13 2022] GET /identity/v3/role_assignments?user.id=376bceb5e33247ec812e693cb2f2e4c7&scope.project.id=eddd1a08322a42ffb5981fd594ba15aa&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:14.147243 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13559d2-8a4c-4b5e-ad7d-36652e18ce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:14.147954 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13559d2-8a4c-4b5e-ad7d-36652e18ce48 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:14.148233 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13559d2-8a4c-4b5e-ad7d-36652e18ce48 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:14.148507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13559d2-8a4c-4b5e-ad7d-36652e18ce48 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:14.149649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13559d2-8a4c-4b5e-ad7d-36652e18ce48 None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:14.151169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13559d2-8a4c-4b5e-ad7d-36652e18ce48 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:14.157792 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 90/185] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:13 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:14.161764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:14.213236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:14.301241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999c941-9a70-4357-b9a8-6ef7eaca7875 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:14.346209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:14.346844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:14.347121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:14.347414 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:14.348155 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] RBAC: Adding query filter params (role.id=e5f3830b373b402693826265019b9344, scope.project.id=eddd1a08322a42ffb5981fd594ba15aa, user.id=376bceb5e33247ec812e693cb2f2e4c7) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:14.348463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:14.349922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc0bfc-27a7-47f1-8551-e1921d0926ca None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:14.356519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 96/186] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:14 2022] GET /identity/v3/role_assignments?user.id=376bceb5e33247ec812e693cb2f2e4c7&scope.project.id=eddd1a08322a42ffb5981fd594ba15aa&role.id=e5f3830b373b402693826265019b9344 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:15.056590 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc6064-527a-4ab4-ab14-91d8a4bbd564 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.057141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc6064-527a-4ab4-ab14-91d8a4bbd564 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.057141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc6064-527a-4ab4-ab14-91d8a4bbd564 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.058764 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 91/187] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:15.064010 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b953f7-725a-417e-ac5c-fd3af6eaa15b None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.064301 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b953f7-725a-417e-ac5c-fd3af6eaa15b None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.064553 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b953f7-725a-417e-ac5c-fd3af6eaa15b None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.064804 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.065233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.065420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.065882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 92/188] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:15.070762 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.071262 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.071789 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.084274 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:15.085468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:15.089872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:15.091013 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:15.153779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 97/189] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:15.167334 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 93/190] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:15.425750 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:15.430447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:15.433795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:15.449409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:15.534983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304902ae-06ea-41a7-a2d6-8bb917ea21be None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:15.538386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776ba25-92aa-4453-9fa6-f05baa839e66 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:15.571241 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:15.571922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.572203 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.572476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.575590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:15.576233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.576519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.576774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.577932 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:15.579386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:15.582821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 98/191] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:15.583582 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:15.586215 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:15.588577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:15.590084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:15.590926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 94/192] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:15.596141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:15.664856 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:15.682305 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:15.694366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf6ca-18c1-4914-99c6-a21737d44fb4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:15.704820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b2529-5381-4972-becf-a1f60d451b9b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:15.734519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5675-7af8-4a4f-9b9a-e067649325fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:15.735584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5675-7af8-4a4f-9b9a-e067649325fb None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.736112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5675-7af8-4a4f-9b9a-e067649325fb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.736598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5675-7af8-4a4f-9b9a-e067649325fb None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.737473 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5675-7af8-4a4f-9b9a-e067649325fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:15.739125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5675-7af8-4a4f-9b9a-e067649325fb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:15.743529 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:15.744150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.744434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.744713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.745387 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:15.745691 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:15.747057 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:15.756115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 95/193] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:15.761664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:15.833887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 99/194] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:15 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:00:15.833887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e37d1-29b7-4ba9-aba1-ee3d4226b75c None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.833887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e37d1-29b7-4ba9-aba1-ee3d4226b75c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.833887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e37d1-29b7-4ba9-aba1-ee3d4226b75c None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.833887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 100/195] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:15.835620 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.835913 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.836162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.855467 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:15.856706 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:15.871687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4cc2-1e67-40cd-b91c-7b95408a52a8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:15.905920 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:15.915125 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:15.915804 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:15.916143 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:15.916359 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] PATH_INFO: `/v3/users/5367f804ca04460cb89cc8845aee20be` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:15.933077 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] RBAC: Authorizing `identity:get_user(user_id=5367f804ca04460cb89cc8845aee20be)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:15.935347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 101/196] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:15.935492 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:15.936567 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 96/197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:15 2022] GET /identity/v3/users/5367f804ca04460cb89cc8845aee20be => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:15.941740 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:16.022451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:16.047499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0876b0-aac4-4134-834c-bae958bb6e08 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:16.083001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:16.083684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:16.083940 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:16.084209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:16.085336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:16.086783 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:16.088903 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 102/198] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:15 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:16.094366 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:16.204012 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551d610-63ee-4522-a4ba-fd1a84cb439c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:16.211746 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:16.245620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:16.246475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:16.247103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:16.247416 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/5367f804ca04460cb89cc8845aee20be/roles/75b9861121374bc2a5fcb0cee200c294` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:16.250244 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddd1a08322a42ffb5981fd594ba15aa, user_id=5367f804ca04460cb89cc8845aee20be, role_id=75b9861121374bc2a5fcb0cee200c294)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:16.251863 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:16.327781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 97/199] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:16 2022] PUT /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/users/5367f804ca04460cb89cc8845aee20be/roles/75b9861121374bc2a5fcb0cee200c294 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:16.337236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f75d3-f98a-4e24-9926-4087772edce2 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:16.381417 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:16.382518 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:16.382990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:16.383517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] PATH_INFO: `/v3/roles/member` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:16.397137 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:16.402908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:16.403996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:16.404910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:16.412623 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 103/200] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:16.414812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:16.466832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:16.527117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88116742-0eea-45ec-8f9d-69e0c3b4d669 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:16.567333 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:16.567977 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:16.568256 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:16.568533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:16.569207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:16.569511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:16.570886 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:16.580193 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 98/201] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:16.586057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:16.589611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:16.693972 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459def3e-b2ad-4302-bd0d-3a92480f626a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:16.730435 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:16.731000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:16.731283 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:16.731599 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] PATH_INFO: `/v3/users/f58c5f4aee044c908131c3e9928a8a2d` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:16.732821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] RBAC: Authorizing `identity:get_user(user_id=f58c5f4aee044c908131c3e9928a8a2d)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:16.734820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:16.736526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 104/202] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:16 2022] GET /identity/v3/users/f58c5f4aee044c908131c3e9928a8a2d => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:16.742126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:16.854339 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cc5f3-d8c1-41af-870b-3e1794fb43f1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:16.862231 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:16.893297 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:16.893876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:16.894157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:16.894490 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:16.966250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:16.967794 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:17.075660 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 99/203] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 06:00:17.077208 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:17.150331 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:17.183311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf3ff4-433d-478d-890f-678cae53c69f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:17.218584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321648f-bdca-44ee-9e8a-e03c2280ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:17.219226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321648f-bdca-44ee-9e8a-e03c2280ca9a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:17.219536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321648f-bdca-44ee-9e8a-e03c2280ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:17.219812 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321648f-bdca-44ee-9e8a-e03c2280ca9a None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:17.221015 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321648f-bdca-44ee-9e8a-e03c2280ca9a None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:17.222481 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321648f-bdca-44ee-9e8a-e03c2280ca9a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:17.224470 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 105/204] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:17 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:17.230380 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:17.238905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:17.338275 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235e0d7-b84d-4e75-a62d-e158d2c489ad None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:17.374328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:17.374943 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:17.375223 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:17.375545 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:17.376318 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] RBAC: Adding query filter params (role.id=abebdef18b5648ea9d768c2b83eb2ddd, scope.project.id=eddd1a08322a42ffb5981fd594ba15aa, user.id=f58c5f4aee044c908131c3e9928a8a2d) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:17.376632 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:17.378067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708aec7-e681-4c90-bb7d-02ff705a05f8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:17.384552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 100/205] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:17 2022] GET /identity/v3/role_assignments?user.id=f58c5f4aee044c908131c3e9928a8a2d&scope.project.id=eddd1a08322a42ffb5981fd594ba15aa&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:18.165853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c829169-6433-4f51-b49d-50533fe546e4 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.166426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c829169-6433-4f51-b49d-50533fe546e4 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.166426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c829169-6433-4f51-b49d-50533fe546e4 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.167879 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 106/206] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:18.174222 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900750c-4f16-4ec7-9374-a590e8037fca None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.174549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900750c-4f16-4ec7-9374-a590e8037fca None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.174790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900750c-4f16-4ec7-9374-a590e8037fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.194724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-2900750c-4f16-4ec7-9374-a590e8037fca None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:18.195957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-2900750c-4f16-4ec7-9374-a590e8037fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:18.259105 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:18.264710 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 101/207] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:18.281421 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be4902-f78a-47d1-b6de-80f91d9e2f6c None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.281805 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be4902-f78a-47d1-b6de-80f91d9e2f6c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.282034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be4902-f78a-47d1-b6de-80f91d9e2f6c None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.283513 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 107/208] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:18.288552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea45193-6f6f-4a89-a54e-0b5a6564aa4f None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.288835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea45193-6f6f-4a89-a54e-0b5a6564aa4f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.289103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea45193-6f6f-4a89-a54e-0b5a6564aa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.308227 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-7ea45193-6f6f-4a89-a54e-0b5a6564aa4f None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:18.309396 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-7ea45193-6f6f-4a89-a54e-0b5a6564aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:18.375020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 102/209] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:18.532727 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be4902-f78a-47d1-b6de-80f91d9e2f6c None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:18.595394 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54035f-a442-4ef9-b21f-368d61d29acd None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.595662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54035f-a442-4ef9-b21f-368d61d29acd None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.595915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54035f-a442-4ef9-b21f-368d61d29acd None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.597389 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 103/210] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:00:18.602454 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.602746 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.603021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.621626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:18.622822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:18.639105 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be4902-f78a-47d1-b6de-80f91d9e2f6c None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:18.673660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:18.674291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.674569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.674840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.681676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:18.685208 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:18.690310 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 104/211] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:18.690448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:18.694406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:18.695346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:18.696091 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:18.696167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 108/212] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:18.696985 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:18.761771 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:18.800491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42454c-c1ed-4186-9a08-729390e848d4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:18.809446 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffaa05-ce35-4abe-9046-87fd343d0cf7 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:18.839667 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:18.841204 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.841701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.842160 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.843619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:18.844129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:18.845348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:18.845920 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:18.846041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:18.846249 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:18.846522 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:18.847649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:18.849219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:18.850636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 105/213] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:18.856010 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:18.856185 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 109/214] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:18.861658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:18.923508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:18.965533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b00b16-8043-4bcd-829e-1732d9b106b6 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:18.966504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db4ebc-b013-4b70-b333-aae07c2bf2f9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:18.992476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:18.999671 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.000299 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.000441 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.000566 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.000834 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] PATH_INFO: `/v3/users/cbc2e698cd7140dbb7ac7b90d14b0e45` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.001050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.001332 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.001602 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.002259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:19.003729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:19.014086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cbc2e698cd7140dbb7ac7b90d14b0e45)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:19.016071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:19.017694 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 110/215] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:18 2022] GET /identity/v3/users/cbc2e698cd7140dbb7ac7b90d14b0e45 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:19.018456 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:19.060114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 106/216] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:18 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:00:19.060849 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:19.124831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d2408-d26c-4a99-9730-9c63f26c45f8 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:19.144420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:19.165392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.166352 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.166925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.166925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.171655 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bd34f-d1e2-4661-ac22-bb7112bbfe8c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:19.180054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:19.185411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:19.186451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:19.187395 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 111/217] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:19.192813 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:19.213115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.213753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.214038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.214312 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.215068 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] RBAC: Adding query filter params (role.id=abebdef18b5648ea9d768c2b83eb2ddd, user.id=cbc2e698cd7140dbb7ac7b90d14b0e45, scope.system=all) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:19.215411 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:19.216899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:19.223101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 107/218] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 06:00:19 2022] GET /identity/v3/role_assignments?user.id=cbc2e698cd7140dbb7ac7b90d14b0e45&scope.system=all&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:19.322939 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee7c45-d1ec-440d-b058-24b1d5ff34bc None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:19.366669 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.367341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.367618 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.367892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.368585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] RBAC: Adding query filter params (name=reader) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:19.368898 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:19.370313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:19.381173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 112/219] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:19.387107 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:19.443195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:19.507970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce3f2c-2679-4d4a-9d57-b04fab8def8e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:19.544456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.545435 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.545876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.546874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] PATH_INFO: `/v3/users/5367f804ca04460cb89cc8845aee20be` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.548214 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] RBAC: Authorizing `identity:get_user(user_id=5367f804ca04460cb89cc8845aee20be)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:19.549828 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:19.552002 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 108/220] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:19 2022] GET /identity/v3/users/5367f804ca04460cb89cc8845aee20be => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:19.560448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:19.604451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:19.666987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680d32d-ad5b-4151-a5aa-7ee2dcd03aac None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:19.703903 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:19.704526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:19.704806 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:19.705080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:19.777508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:19.779097 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:19.822021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:19.886388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 113/221] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:19.892139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:20.001447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474827-c141-4c14-9fd0-4382aeed4d59 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:20.018312 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:20.037742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453843ed-8988-47f7-9c55-474e752fe359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:20.038389 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453843ed-8988-47f7-9c55-474e752fe359 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:20.038667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453843ed-8988-47f7-9c55-474e752fe359 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:20.038937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453843ed-8988-47f7-9c55-474e752fe359 None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:20.040386 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453843ed-8988-47f7-9c55-474e752fe359 None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:20.041964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453843ed-8988-47f7-9c55-474e752fe359 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:20.043637 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 109/222] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:19 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:20.050984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:20.166661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c809-e131-4348-9385-7a87486e4ad5 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:20.203023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:20.204034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:20.204524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:20.205002 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:20.205947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] RBAC: Adding query filter params (role.id=75b9861121374bc2a5fcb0cee200c294, scope.project.id=eddd1a08322a42ffb5981fd594ba15aa, user.id=5367f804ca04460cb89cc8845aee20be) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:20.206459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:20.208153 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c210aa-7852-4385-8663-f95f9dc37c42 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:20.214508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 114/223] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:20 2022] GET /identity/v3/role_assignments?user.id=5367f804ca04460cb89cc8845aee20be&scope.project.id=eddd1a08322a42ffb5981fd594ba15aa&role.id=75b9861121374bc2a5fcb0cee200c294 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:21.396221 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afc8d-47ad-4c08-adb8-022eb19a7be7 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.396753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afc8d-47ad-4c08-adb8-022eb19a7be7 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.396753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834afc8d-47ad-4c08-adb8-022eb19a7be7 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.398131 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 110/224] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:21.403315 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9359a-4fe5-4700-9fdf-6d0d3c3e4dba None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.403704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9359a-4fe5-4700-9fdf-6d0d3c3e4dba None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.403880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9359a-4fe5-4700-9fdf-6d0d3c3e4dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.411723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:21.411881 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c17c1a-7964-42ae-8d91-b46b245a39b8 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.412173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c17c1a-7964-42ae-8d91-b46b245a39b8 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.412455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c17c1a-7964-42ae-8d91-b46b245a39b8 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.413784 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 111/225] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:21.418801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42d51e-e4e0-4e7c-8aff-9caf827f81bc None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.419091 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42d51e-e4e0-4e7c-8aff-9caf827f81bc None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.419638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42d51e-e4e0-4e7c-8aff-9caf827f81bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.423724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-4da9359a-4fe5-4700-9fdf-6d0d3c3e4dba None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:21.424914 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-4da9359a-4fe5-4700-9fdf-6d0d3c3e4dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:21.438387 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-bb42d51e-e4e0-4e7c-8aff-9caf827f81bc None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:21.439601 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-bb42d51e-e4e0-4e7c-8aff-9caf827f81bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:21.514376 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 112/226] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:21.517067 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 115/227] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:21.581488 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70c565-ace2-4ef4-8122-ca6bfe7f1a01 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.581795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70c565-ace2-4ef4-8122-ca6bfe7f1a01 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.582048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70c565-ace2-4ef4-8122-ca6bfe7f1a01 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.583576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 113/228] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:21.588677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.588970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.589248 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.607898 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:21.609702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:21.680308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 116/229] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:21.784004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70c565-ace2-4ef4-8122-ca6bfe7f1a01 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:21.792296 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:21.857330 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:21.892957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70c565-ace2-4ef4-8122-ca6bfe7f1a01 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:21.897783 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3567e-f708-4f94-b3f1-93204d1788e8 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:21.909286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:21.930467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:21.931033 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.931598 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.931869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.934402 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:21.935038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:21.935336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:21.935619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:21.942508 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:21.946420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:21.947349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:21.948289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:21.949296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 114/230] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:21.952890 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:21.955342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:21.956465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 117/231] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:21.959583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.052028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0863-92ba-47eb-a079-50b8a5c992e0 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.055952 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651942e4-efb8-4665-b35d-f8e35a8fef99 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.079121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:22.086180 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.088648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.088648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.088648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.088648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.089970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.090514 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.091115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.091428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.091703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.091893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 115/232] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:22.092370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:22.092674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.092837 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.094282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.103274 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 118/233] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:22.103274 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.142451 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:22.197849 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47210b99-bf3f-49a4-b121-6d6b8c8063ef None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.201065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6d15-ac4e-4c90-9923-f4af2753f515 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.231581 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cd004-b858-4192-9400-125c16706525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.232193 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cd004-b858-4192-9400-125c16706525 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.232477 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cd004-b858-4192-9400-125c16706525 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.232749 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cd004-b858-4192-9400-125c16706525 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.233485 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cd004-b858-4192-9400-125c16706525 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:22.233797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cd004-b858-4192-9400-125c16706525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.234141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.234739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.235023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.235163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cd004-b858-4192-9400-125c16706525 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.235328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] PATH_INFO: `/v3/groups` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.235944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.237358 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.243119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 116/234] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:22.243944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cd004-b858-4192-9400-125c16706525 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.310935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 119/235] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Oct 17 06:00:22 2022] POST /identity/v3/groups => generated 229 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:22.312289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.352091 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cd004-b858-4192-9400-125c16706525 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.354780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:22.365817 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:22.392694 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.393388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.393667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.393941 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] PATH_INFO: `/v3/users/cbc2e698cd7140dbb7ac7b90d14b0e45` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.395157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] RBAC: Authorizing `identity:get_user(user_id=cbc2e698cd7140dbb7ac7b90d14b0e45)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.396658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.398329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 117/236] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:22 2022] GET /identity/v3/users/cbc2e698cd7140dbb7ac7b90d14b0e45 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:22.404338 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.428474 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320824-59b1-4c3e-84db-bfb541c19269 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.474171 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.474890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.475182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.475529 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] PATH_INFO: `/v3/users/12be057e00884bce8b0b455d41260141` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.495330 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] RBAC: Authorizing `identity:get_user(user_id=12be057e00884bce8b0b455d41260141)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.497585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.500385 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 120/237] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:22 2022] GET /identity/v3/users/12be057e00884bce8b0b455d41260141 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:22.505906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:22.528240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b968380-9b2c-4d99-9541-d775e5f5182f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.565867 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7327b-4ea5-40ef-b04f-1526f7e02660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.566531 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7327b-4ea5-40ef-b04f-1526f7e02660 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.566809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7327b-4ea5-40ef-b04f-1526f7e02660 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.567091 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7327b-4ea5-40ef-b04f-1526f7e02660 None admin] PATH_INFO: `/v3/system/users/cbc2e698cd7140dbb7ac7b90d14b0e45/roles/abebdef18b5648ea9d768c2b83eb2ddd` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.568534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7327b-4ea5-40ef-b04f-1526f7e02660 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cbc2e698cd7140dbb7ac7b90d14b0e45, role_id=abebdef18b5648ea9d768c2b83eb2ddd)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.569904 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7327b-4ea5-40ef-b04f-1526f7e02660 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.620956 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5576033-9f9b-45f5-bfba-46bbbb63a8bf None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:22.623318 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:22.654401 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:22.654997 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:22.655314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:22.655598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:22.656317 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] RBAC: Adding query filter params (role.id=75b9861121374bc2a5fcb0cee200c294, user.id=12be057e00884bce8b0b455d41260141, scope.system=all) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:22.656619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:22.658063 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963240c-3308-4caa-9ed1-bf53bc285b7c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:22.662883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 121/238] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 06:00:22 2022] GET /identity/v3/role_assignments?user.id=12be057e00884bce8b0b455d41260141&scope.system=all&role.id=75b9861121374bc2a5fcb0cee200c294 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:22.695569 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 118/239] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Oct 17 06:00:22 2022] PUT /identity/v3/system/users/cbc2e698cd7140dbb7ac7b90d14b0e45/roles/abebdef18b5648ea9d768c2b83eb2ddd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:00:24.509619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28aa5e2-66ee-4e35-ac4a-19d5a895fbfc None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:24.510158 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28aa5e2-66ee-4e35-ac4a-19d5a895fbfc None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:24.510158 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28aa5e2-66ee-4e35-ac4a-19d5a895fbfc None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:24.511686 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 122/240] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:24.516726 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc3f2f-5b99-49b5-88be-f002a52f3066 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:24.517120 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc3f2f-5b99-49b5-88be-f002a52f3066 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:24.517283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc3f2f-5b99-49b5-88be-f002a52f3066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:24.535744 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-08bc3f2f-5b99-49b5-88be-f002a52f3066 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:24.536929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-08bc3f2f-5b99-49b5-88be-f002a52f3066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:24.567357 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:24.605411 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 119/241] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:24.834431 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec7bbb-81ca-4f8d-8107-8482e9230062 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:24.834619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec7bbb-81ca-4f8d-8107-8482e9230062 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:24.834892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec7bbb-81ca-4f8d-8107-8482e9230062 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:24.836550 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 123/242] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:24.841007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ac20-55b9-4be8-94df-bb323feaa3ea None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:24.841325 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:24.841433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ac20-55b9-4be8-94df-bb323feaa3ea None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:24.841613 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:24.841721 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408ac20-55b9-4be8-94df-bb323feaa3ea None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:24.841875 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:24.843685 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 120/243] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:24.848788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:24.849070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:24.849340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:24.862322 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:24.863579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:24.868845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:24.870002 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:24.934492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 124/244] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:24.935377 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:24.938687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 121/245] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:00:25.032119 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:25.035935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f324f5-ed00-4e6c-a047-72c087732f09 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.070368 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.071001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.071283 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.071582 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.072685 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.074197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.075940 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 125/246] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:25.081386 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.174362 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:25.180665 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078be6f-a390-4d9a-a7b2-e85caacd7903 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.201218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.216249 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.216861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.217139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.217409 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] PATH_INFO: `/v3/groups` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.218121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.219575 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.287965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 122/247] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Oct 17 06:00:25 2022] POST /identity/v3/groups => generated 226 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:25.288791 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.305695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298047-fdba-49ef-8c62-ae970d4e0428 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.346032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.346696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.346982 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.347260 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.359676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:25.364788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:25.366020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:25.367041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 126/248] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:25.372066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.402584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:25.424591 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ce919-75c2-4519-b5bf-50eaaf11eb69 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.461852 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.462507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.463212 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.463550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.477724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:25.484780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:25.485806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:25.486674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 123/249] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:25.490785 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.496156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f561-b763-4c4b-a3ed-e342db89a56e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.512349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:25.539105 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.539773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.540029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.540311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.540990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:25.541299 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.542789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.555840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 127/250] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:25.561884 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.620289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca67fb8-8d8f-4b43-902c-ed51196153eb None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.664523 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.668832 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:25.668986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.668986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.668986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.668986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:25.668986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.668986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.679950 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 124/251] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:25.688059 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.690440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c91f8a-de2b-4549-83f4-4358ba017ed3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.731615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.732262 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.732538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.732811 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] PATH_INFO: `/v3/users/12be057e00884bce8b0b455d41260141` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.734509 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] RBAC: Authorizing `identity:get_user(user_id=12be057e00884bce8b0b455d41260141)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.735981 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.738027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 128/252] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/users/12be057e00884bce8b0b455d41260141 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:25.743556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.790235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:25.818219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12b56-f184-47ab-80c9-751198442bd8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.864596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.865247 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.865363 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7428270-2364-4614-8716-8dc311fd2a2a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:25.865560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.865798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] PATH_INFO: `/v3/users/cbc2e698cd7140dbb7ac7b90d14b0e45` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.867207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] RBAC: Authorizing `identity:get_user(user_id=cbc2e698cd7140dbb7ac7b90d14b0e45)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.869667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.871379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 125/253] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/users/cbc2e698cd7140dbb7ac7b90d14b0e45 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:25.877479 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:25.907696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca079751-0fdf-4221-bf75-215b84625b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:25.908401 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079751-0fdf-4221-bf75-215b84625b1b None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:25.908686 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079751-0fdf-4221-bf75-215b84625b1b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:25.909013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079751-0fdf-4221-bf75-215b84625b1b None admin] PATH_INFO: `/v3/system/users/12be057e00884bce8b0b455d41260141/roles/75b9861121374bc2a5fcb0cee200c294` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:25.910611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca079751-0fdf-4221-bf75-215b84625b1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=12be057e00884bce8b0b455d41260141, role_id=75b9861121374bc2a5fcb0cee200c294)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:25.912174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca079751-0fdf-4221-bf75-215b84625b1b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:25.980712 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 129/254] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Oct 17 06:00:25 2022] PUT /identity/v3/system/users/12be057e00884bce8b0b455d41260141/roles/75b9861121374bc2a5fcb0cee200c294 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:00:26.006270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d91ebd-2104-42a9-bfd3-df6e0492cf4e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:26.046362 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:26.047000 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:26.047360 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:26.047641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:26.048392 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] RBAC: Adding query filter params (role.id=abebdef18b5648ea9d768c2b83eb2ddd, user.id=cbc2e698cd7140dbb7ac7b90d14b0e45, scope.system=all) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:26.048700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:26.050233 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390978c-3930-4d3c-810a-0c635213cb31 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:26.065528 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 126/255] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 06:00:25 2022] GET /identity/v3/role_assignments?user.id=cbc2e698cd7140dbb7ac7b90d14b0e45&scope.system=all&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:27.712010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630ae3b-c7a6-45bf-a82a-786902109da4 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:27.712557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630ae3b-c7a6-45bf-a82a-786902109da4 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:27.712557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630ae3b-c7a6-45bf-a82a-786902109da4 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:27.714247 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 130/256] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:27.725012 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:27.725459 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:27.725538 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:27.734103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:27.748215 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:27.750006 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:27.835417 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 127/257] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:00:28.119145 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630ae3b-c7a6-45bf-a82a-786902109da4 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:28.133200 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630ae3b-c7a6-45bf-a82a-786902109da4 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:28.138086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:28.138734 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.139008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.139283 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] PATH_INFO: `/v3/roles/member` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.151739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:28.157170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:28.158114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:28.158991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:28.160537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 131/258] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:00:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:28.164666 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:28.174194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638acaa-e2d0-4f38-a0e5-d6a86f190121 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:28.180520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:28.181470 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.181748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.182019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.182705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] RBAC: Adding query filter params (name=member) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:28.183002 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:28.184815 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: WARNING py.warnings [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:00:28.184815 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: warnings.warn(msg) Oct 17 06:00:28.184815 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: Oct 17 06:00:28.185711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:28.199360 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 128/259] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:00:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:28.204252 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:28.214423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de01c6f-c7e6-4815-b953-4451dac6697d admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:28.219531 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:28.220163 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.220443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.220718 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.225359 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:28.229461 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:28.231313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: WARNING py.warnings [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:00:28.231313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: warnings.warn(msg) Oct 17 06:00:28.231313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: Oct 17 06:00:28.232555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:28.234293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 132/260] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 06:00:28 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:28.251115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:28.261714 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7deafc-fb5d-4df5-940e-d8154d087193 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:28.272659 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbd3e5-0dfd-45f4-825e-4fd2642137d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:28.273355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbd3e5-0dfd-45f4-825e-4fd2642137d7 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.273635 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbd3e5-0dfd-45f4-825e-4fd2642137d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.273912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbd3e5-0dfd-45f4-825e-4fd2642137d7 admin admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.275123 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbd3e5-0dfd-45f4-825e-4fd2642137d7 admin admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:28.277321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbd3e5-0dfd-45f4-825e-4fd2642137d7 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:28.280898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 129/261] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 06:00:28 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:28.285863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:28.298051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d56f-8cf7-4669-b2b0-25728d1cd7f5 admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:28.302662 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:28.303335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.303623 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.303896 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/abebdef18b5648ea9d768c2b83eb2ddd` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.309689 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d062649850c4cc280349e14d6730b23, group_id=fc1fab04a1c24ad1bb4dfe03e1acb488, role_id=abebdef18b5648ea9d768c2b83eb2ddd)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:28.310753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: WARNING py.warnings [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:00:28.310753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: warnings.warn(msg) Oct 17 06:00:28.310753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: Oct 17 06:00:28.313410 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ef88-8adf-4e1f-a4dc-cb8b8c20e43a admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:28.426855 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 133/262] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 06:00:28 2022] PUT /identity/v3/projects/8d062649850c4cc280349e14d6730b23/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/abebdef18b5648ea9d768c2b83eb2ddd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:28.698457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74dd1-40ae-42ee-a293-28e78eb9fe6c None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.698643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74dd1-40ae-42ee-a293-28e78eb9fe6c None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.698920 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74dd1-40ae-42ee-a293-28e78eb9fe6c None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.700718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 130/263] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:28.710607 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea3501-6c2d-4ea5-ae05-fd17b5eb7f98 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.710957 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea3501-6c2d-4ea5-ae05-fd17b5eb7f98 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.711154 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea3501-6c2d-4ea5-ae05-fd17b5eb7f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.736584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-d6ea3501-6c2d-4ea5-ae05-fd17b5eb7f98 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:28.737865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-d6ea3501-6c2d-4ea5-ae05-fd17b5eb7f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:28.796793 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37861e-7cde-4921-91f3-8f4c6032722a None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.797336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37861e-7cde-4921-91f3-8f4c6032722a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.797336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37861e-7cde-4921-91f3-8f4c6032722a None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.798817 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 131/264] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:00:28.804395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.804674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.805005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.829658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 134/265] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:00:28.835425 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:28.838059 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:28.838526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042c5a1-05a2-4a32-b5cc-a1359934cb53 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.838877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042c5a1-05a2-4a32-b5cc-a1359934cb53 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.839150 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042c5a1-05a2-4a32-b5cc-a1359934cb53 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.840897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 135/266] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:28.849301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:28.849607 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:28.856687 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:28.856855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 132/267] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:00:28.876077 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:28.877389 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:28.959784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 136/268] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:29.309395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.318911 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1af37-cf02-4107-8e6d-f8b6f3277982 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.327899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.328551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.328828 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.329099 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.332706 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.348228 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:29.349540 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1db22-9d32-45a1-bda3-5ab94e8f5b91 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:29.353165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:29.354237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:29.355152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 133/269] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:29.357857 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.359041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.359342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.359624 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.360744 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.373526 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb7ce8-a920-4b44-96fe-d5617eab4697 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.380298 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:29.385297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:29.387525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:29.390122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 137/270] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:29.395925 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.395925 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.395925 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.395925 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.396596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] RBAC: Adding query filter params (name=member) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:29.396596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.396596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.398701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.409476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 134/271] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:29.414932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.417817 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52b393-b85d-4024-a089-e16020fc62b2 admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.428448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33f1fa-472a-4635-b345-5a68372233c8 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.431630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.432381 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.432666 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.432941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.433632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:29.433937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.434958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: WARNING py.warnings [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:00:29.434958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: warnings.warn(msg) Oct 17 06:00:29.434958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: Oct 17 06:00:29.435945 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.447987 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.448644 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.448920 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.449187 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.450361 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.451383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: WARNING py.warnings [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:00:29.451383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: warnings.warn(msg) Oct 17 06:00:29.451383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: Oct 17 06:00:29.453122 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.454936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 138/273] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:29.455128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 135/273] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:29.470150 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.471104 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.485766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd1529-5857-4e7b-903e-931e1df34859 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.489192 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4ded7-12b5-4c6b-a040-42afcd86ebe0 admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.500450 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.501092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.501367 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.501638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.503960 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.504613 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.504904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.505185 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.506365 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.507959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.508520 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.509866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.510173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 139/274] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:29.511751 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 136/275] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:29.517826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.518050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.535114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f43b8-a3b8-4a59-9642-cf81bfe0f2b5 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.537890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50c70e-7423-4395-be83-3b2c8df08632 admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.541732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.542403 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.542680 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.542949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/abebdef18b5648ea9d768c2b83eb2ddd` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.545009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddd1a08322a42ffb5981fd594ba15aa, group_id=fc1fab04a1c24ad1bb4dfe03e1acb488, role_id=abebdef18b5648ea9d768c2b83eb2ddd)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.546735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: WARNING py.warnings [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 06:00:29.546735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: warnings.warn(msg) Oct 17 06:00:29.546735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: Oct 17 06:00:29.548720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.555997 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.556891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.557174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.557447 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.558605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.560741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.562892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 140/276] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:29.563900 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.600950 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:29.631383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 137/277] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 06:00:29 2022] PUT /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/abebdef18b5648ea9d768c2b83eb2ddd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:00:29.632202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.724237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdafa6-52eb-4bd4-ad38-e6547bbc0e7e admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.750930 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d71799-1402-4944-b462-a658b793a2d7 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:29.761599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.762287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.762565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.762835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/e5f3830b373b402693826265019b9344` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.765134 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d062649850c4cc280349e14d6730b23, group_id=fc1fab04a1c24ad1bb4dfe03e1acb488, role_id=e5f3830b373b402693826265019b9344)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:29.767840 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:29.805724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:29.806645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:29.806917 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:29.807197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:29.825896 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:29.830810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:29.831895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:29.832779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 141/278] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:29.837935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:29.852605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 138/279] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 06:00:29 2022] PUT /identity/v3/projects/8d062649850c4cc280349e14d6730b23/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/e5f3830b373b402693826265019b9344 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:29.971746 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:29.979477 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97ddb5-c04b-4b93-98d2-70ab5988f23c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.022755 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.023403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.023689 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.023966 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.024651 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:30.024957 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.026366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1d101-bf3a-4613-b952-7c9ddcbdd4d2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:30.037606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 142/280] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:30.043821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.097899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:30.146226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14e9f2-1134-4c09-b61a-8841211f6347 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.146537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14e9f2-1134-4c09-b61a-8841211f6347 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.146793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14e9f2-1134-4c09-b61a-8841211f6347 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.148357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 143/281] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:30.156646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70895097-2ed0-4b41-85e9-ec99f9062826 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.156944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70895097-2ed0-4b41-85e9-ec99f9062826 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.157217 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70895097-2ed0-4b41-85e9-ec99f9062826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.200369 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-70895097-2ed0-4b41-85e9-ec99f9062826 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:30.201610 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-70895097-2ed0-4b41-85e9-ec99f9062826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:30.203935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42093d28-66d3-49a2-bc2b-647cc61a4916 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.260374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.261297 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.261724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.262142 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] PATH_INFO: `/v3/users/12be057e00884bce8b0b455d41260141` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.263928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=12be057e00884bce8b0b455d41260141)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.267985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:30.271407 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 139/282] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:00:30 2022] GET /identity/v3/users/12be057e00884bce8b0b455d41260141 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:30.277530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.360394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 144/283] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 06:00:30.412967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:30.420263 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be2294-0f7c-41c8-9547-86c5e8a0a3c9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.461402 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.462385 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.462868 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.463395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.464401 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] RBAC: Adding query filter params (role.id=75b9861121374bc2a5fcb0cee200c294, user.id=12be057e00884bce8b0b455d41260141, scope.system=all) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:30.464915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.466614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401ca5d-65fc-4456-ab93-a542846d2893 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:30.487891 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 140/284] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Oct 17 06:00:30 2022] GET /identity/v3/role_assignments?user.id=12be057e00884bce8b0b455d41260141&scope.system=all&role.id=75b9861121374bc2a5fcb0cee200c294 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:30.714138 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99577c6-5c96-4bb7-9a94-2a1d36c76355 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.714334 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99577c6-5c96-4bb7-9a94-2a1d36c76355 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.714634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99577c6-5c96-4bb7-9a94-2a1d36c76355 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.716392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 145/285] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:30.722256 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.722555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.722807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.744873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:30.746118 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:30.757961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 141/286] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 06:00:30.802510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99577c6-5c96-4bb7-9a94-2a1d36c76355 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.822023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99577c6-5c96-4bb7-9a94-2a1d36c76355 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.829890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.830574 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.830851 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.831122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.848459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:30.853459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:30.854463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:30.855371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 146/287] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 06:00:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:30.861241 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.872904 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76430652-6801-43ce-a8fd-69d1d0840435 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.879236 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.879895 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.880173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.880442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.881115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:30.881414 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.883001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:30.894296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 142/288] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 06:00:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:30.900104 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.913105 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e65df5-4f0c-439e-877b-cfd3e05f426c admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.918678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.919334 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.919630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.919904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.921580 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.923519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:30.925330 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 147/289] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 06:00:30 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:30.931611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.943915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8841c02-475e-48e2-947d-db3af01e98f9 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.948901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.949557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.949833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.950105 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.951243 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.953857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:30.956836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 143/290] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 06:00:30 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:30.961831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:30.972428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1577ee6-ee47-443e-bd9a-cb6b76f39171 admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:30.981633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:30.982300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:30.982580 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:30.982854 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/e5f3830b373b402693826265019b9344` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:30.988029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddd1a08322a42ffb5981fd594ba15aa, group_id=fc1fab04a1c24ad1bb4dfe03e1acb488, role_id=e5f3830b373b402693826265019b9344)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:30.990727 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:31.078648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 148/291] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 06:00:30 2022] PUT /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa/groups/fc1fab04a1c24ad1bb4dfe03e1acb488/roles/e5f3830b373b402693826265019b9344 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:31.166574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:31.177335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfda99c-6dc2-430e-adf6-2256f9f3834a admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:31.261192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:31.261846 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:31.262133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:31.262408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:31.280591 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:31.286121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:31.287114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:31.288047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:31.295376 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 144/292] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:00:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 121 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:31.296800 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:31.307813 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef1d8-5a49-4b36-b996-6f18f92effdc admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:31.319659 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:31.320795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:31.321287 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:31.321809 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:31.322705 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:31.323330 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:31.325458 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:31.345282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 149/293] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:31.351524 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:31.369300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1867ef-8580-4bae-b4d7-573c2f73db2b admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:31.380498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:31.380498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:31.380781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:31.381027 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] PATH_INFO: `/v3/groups/a902199f76d54ab5981cb65eec0e68e6` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:31.391240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] RBAC: Authorizing `identity:get_group(group_id=a902199f76d54ab5981cb65eec0e68e6)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:31.393249 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:31.395242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 145/294] 10.222.0.13 () {56 vars in 1201 bytes} [Mon Oct 17 06:00:31 2022] GET /identity/v3/groups/a902199f76d54ab5981cb65eec0e68e6 => generated 229 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:31.401394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:31.413522 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798460b7-35ff-4803-bf88-8b66c4546f02 admin admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:31.419918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b605a4-3c59-4439-8d54-9daa12339864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:31.420699 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b605a4-3c59-4439-8d54-9daa12339864 admin admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:31.420988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b605a4-3c59-4439-8d54-9daa12339864 admin admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:31.421357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b605a4-3c59-4439-8d54-9daa12339864 admin admin] PATH_INFO: `/v3/projects/67b5d25652604b029091af61167fe6bf` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:31.423198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b605a4-3c59-4439-8d54-9daa12339864 admin admin] RBAC: Authorizing `identity:get_project(project_id=67b5d25652604b029091af61167fe6bf)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:31.425118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b605a4-3c59-4439-8d54-9daa12339864 admin admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:31.426929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 150/295] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Oct 17 06:00:31 2022] GET /identity/v3/projects/67b5d25652604b029091af61167fe6bf => generated 344 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:31.432987 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:31.444668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da938a0a-7257-4c6e-aa6c-a9745809aa60 admin admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:31.451944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ddba-b003-447e-92cd-d1e15dcc657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67b5d25652604b029091af61167fe6bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:31.452913 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ddba-b003-447e-92cd-d1e15dcc657f admin admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:31.453657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ddba-b003-447e-92cd-d1e15dcc657f admin admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:31.454129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ddba-b003-447e-92cd-d1e15dcc657f admin admin] PATH_INFO: `/v3/projects/67b5d25652604b029091af61167fe6bf/groups/a902199f76d54ab5981cb65eec0e68e6/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:31.456455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ddba-b003-447e-92cd-d1e15dcc657f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b5d25652604b029091af61167fe6bf, group_id=a902199f76d54ab5981cb65eec0e68e6, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:31.459526 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ddba-b003-447e-92cd-d1e15dcc657f admin admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:31.529935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 146/296] 10.222.0.13 () {58 vars in 1463 bytes} [Mon Oct 17 06:00:31 2022] PUT /identity/v3/projects/67b5d25652604b029091af61167fe6bf/groups/a902199f76d54ab5981cb65eec0e68e6/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:32.320711 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a1775-88f2-4c2a-a31e-737a678ac5de None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.321234 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a1775-88f2-4c2a-a31e-737a678ac5de None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.321234 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a1775-88f2-4c2a-a31e-737a678ac5de None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.322841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 151/297] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:32.331076 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161268-9612-442f-ba61-1aa6fb8677f5 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.332031 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161268-9612-442f-ba61-1aa6fb8677f5 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.332510 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161268-9612-442f-ba61-1aa6fb8677f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.360196 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-db161268-9612-442f-ba61-1aa6fb8677f5 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:32.361761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-db161268-9612-442f-ba61-1aa6fb8677f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:32.380704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ef6b-3588-40f5-90f0-ff0b66014e9d None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.381051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ef6b-3588-40f5-90f0-ff0b66014e9d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.381253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ef6b-3588-40f5-90f0-ff0b66014e9d None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.383696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 152/298] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:32.388107 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee1770-37b4-4310-8833-7c70ef33f2ec None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.388107 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee1770-37b4-4310-8833-7c70ef33f2ec None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.388428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee1770-37b4-4310-8833-7c70ef33f2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.432621 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-96ee1770-37b4-4310-8833-7c70ef33f2ec None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:32.435539 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-96ee1770-37b4-4310-8833-7c70ef33f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:32.442622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:32.488489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 147/299] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:32.535003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:32.548047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 153/300] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:32.772997 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667305de-7109-4b90-baf3-09c3e5b0c701 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.773292 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667305de-7109-4b90-baf3-09c3e5b0c701 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.773565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667305de-7109-4b90-baf3-09c3e5b0c701 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.774910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 148/301] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:32.780468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.780753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.781025 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.795683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667305de-7109-4b90-baf3-09c3e5b0c701 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:32.805815 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:32.807403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:32.908675 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 154/302] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:32.941049 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667305de-7109-4b90-baf3-09c3e5b0c701 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:32.977620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:32.978302 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:32.978584 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:32.978853 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:32.989998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:32.994980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:32.995949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:32.996824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 149/303] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:33.002606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.128323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.149054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55af8ff-a33e-4603-b254-f9d1d1b7a564 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.172951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:33.189335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.189957 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.190231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.190501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.191177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] RBAC: Adding query filter params (name=member) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:33.191504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:33.192877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:33.211723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 155/304] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:33.220318 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.224337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:33.307113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1e71d-b3d9-4a6e-9d1f-9ffe246a1902 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.341355 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362069a7-c627-41fd-b9c0-3a4ada117550 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.356124 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.357651 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.358895 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.361493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] PATH_INFO: `/v3/roles/member` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.376233 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:33.382501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.383056 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.383374 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:33.383630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:33.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 06:00:33.385419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:33.386068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.389283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 150/305] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:33.390319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.447730 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:33.480574 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:33.482271 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:33.535389 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ec368-d375-4f55-bb04-b53c146bc959 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.540558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:33.585565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.586924 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.587794 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.588246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.603219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:33.610476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:33.611677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:33.612736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:33.614885 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 151/306] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:33.616273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.624619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 156/307] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:33.625597 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.724600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:33.750621 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d80f5-c9d4-4098-8b8d-adb2b754a86c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.764240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0bc4-5055-4dfd-a5fa-a5dc18cde430 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.789063 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.789678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.789962 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.790237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.790980 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:33.791289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:33.792713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:33.801865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 157/308] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:33.808379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.809064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.809860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.810309 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.810737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.811969 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] RBAC: Adding query filter params (name=member) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:33.813421 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:33.814961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:33.825886 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 152/309] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:33.827525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.867601 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:33.938271 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a12b-0a98-404e-b242-d438f978a056 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.956063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369999e0-f848-46c3-a9db-3a8effa16212 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:33.975085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.975983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.976416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.976868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:33.978445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:33.980064 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:33.981842 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 158/310] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:33.982766 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:33.993580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:33.994211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:33.994494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:33.994770 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.000302 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.082064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.084113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.099968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a041f01-5d79-4f0c-9665-a250e4dc5528 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.115563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.136427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39144f27-0be1-4bb7-aea0-47d70b5a6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.137774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39144f27-0be1-4bb7-aea0-47d70b5a6e82 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.137774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39144f27-0be1-4bb7-aea0-47d70b5a6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.137986 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39144f27-0be1-4bb7-aea0-47d70b5a6e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.206852 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 153/311] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:34.208053 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.232140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39144f27-0be1-4bb7-aea0-47d70b5a6e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.234432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39144f27-0be1-4bb7-aea0-47d70b5a6e82 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.252310 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.334720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ddf47-7092-4134-9671-f06b247573a7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.368503 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.374983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 159/312] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:34.378905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd503-75fe-4261-8226-1628468fad28 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.378905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd503-75fe-4261-8226-1628468fad28 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.378905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd503-75fe-4261-8226-1628468fad28 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.380016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 160/313] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:34.397391 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd503-75fe-4261-8226-1628468fad28 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.397540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.397540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.397540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.397540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.397833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.397833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.397833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 154/314] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:34.397833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.517572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75e78-8d63-4fdd-b72a-6dc41f2e7046 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.518972 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd503-75fe-4261-8226-1628468fad28 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.535239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.559087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.559502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4d78d-cfe0-40f6-bdac-7dcc589cea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.560141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d78d-cfe0-40f6-bdac-7dcc589cea5d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.560425 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d78d-cfe0-40f6-bdac-7dcc589cea5d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.560567 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.560702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d78d-cfe0-40f6-bdac-7dcc589cea5d None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.560790 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.561003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] PATH_INFO: `/v3/projects/8d062649850c4cc280349e14d6730b23` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.562560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4d78d-cfe0-40f6-bdac-7dcc589cea5d None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.563282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] RBAC: Authorizing `identity:get_project(project_id=8d062649850c4cc280349e14d6730b23)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.564063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4d78d-cfe0-40f6-bdac-7dcc589cea5d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.565358 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.565900 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 155/315] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:34.567048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3bdbc-485a-4e07-9041-a9edc82fb0bc None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.567157 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 161/316] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/projects/8d062649850c4cc280349e14d6730b23 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:34.568315 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.571599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3bdbc-485a-4e07-9041-a9edc82fb0bc None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.571919 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3bdbc-485a-4e07-9041-a9edc82fb0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.592752 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-41d3bdbc-485a-4e07-9041-a9edc82fb0bc None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:34.594224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-41d3bdbc-485a-4e07-9041-a9edc82fb0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:34.662754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.675681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 156/317] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:34.676929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba342a4-a1bf-4dbb-a77a-2d9b6ba237ad None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.677206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba342a4-a1bf-4dbb-a77a-2d9b6ba237ad None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.677471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba342a4-a1bf-4dbb-a77a-2d9b6ba237ad None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.678717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 157/318] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:34.679431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba342a4-a1bf-4dbb-a77a-2d9b6ba237ad None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.695520 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c27bd-72d3-43de-bed6-1facdad82945 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.738125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.738738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.739019 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.739328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.740074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] RBAC: Adding query filter params (group.id=fc1fab04a1c24ad1bb4dfe03e1acb488, role.id=abebdef18b5648ea9d768c2b83eb2ddd, scope.project.id=8d062649850c4cc280349e14d6730b23) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:34.740384 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.741837 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.748388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 162/319] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/role_assignments?group.id=fc1fab04a1c24ad1bb4dfe03e1acb488&scope.project.id=8d062649850c4cc280349e14d6730b23&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:34.749257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.806150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba342a4-a1bf-4dbb-a77a-2d9b6ba237ad None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.831726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.850911 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b874e-b5df-47b5-9a0b-14af47973932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.852211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b874e-b5df-47b5-9a0b-14af47973932 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.852700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b874e-b5df-47b5-9a0b-14af47973932 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.853177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b874e-b5df-47b5-9a0b-14af47973932 None admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.854723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b874e-b5df-47b5-9a0b-14af47973932 None admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.857099 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b874e-b5df-47b5-9a0b-14af47973932 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.859583 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 158/320] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:34.861164 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.861702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.862174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.883857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:34.885516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:34.887618 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe23556-45b5-4066-9698-b7c0d2a339a4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:34.935286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:34.935286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:34.935286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:34.935679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:34.937077 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:34.938630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:34.941001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 163/321] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:34.941816 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:34.967041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:34.987059 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 159/322] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:34.988175 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.072889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac9d6-4d22-4016-af8f-2cd30134af4f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.089788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.113529 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.118897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.118897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.118897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.118897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] RBAC: Adding query filter params (group.id=fc1fab04a1c24ad1bb4dfe03e1acb488, role.id=abebdef18b5648ea9d768c2b83eb2ddd, scope.project.id=eddd1a08322a42ffb5981fd594ba15aa) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:35.119214 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538e46e-968f-49ba-ab39-8cb2593f54a4 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.119411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:35.119411 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:35.125412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 164/323] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/role_assignments?group.id=fc1fab04a1c24ad1bb4dfe03e1acb488&scope.project.id=eddd1a08322a42ffb5981fd594ba15aa&role.id=abebdef18b5648ea9d768c2b83eb2ddd => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:35.126349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.166286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.167093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.167518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.167885 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.168840 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] RBAC: Adding query filter params (group.id=fc1fab04a1c24ad1bb4dfe03e1acb488, role.id=e5f3830b373b402693826265019b9344, scope.project.id=8d062649850c4cc280349e14d6730b23) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:35.169254 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:35.171209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:35.178750 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 160/324] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 06:00:34 2022] GET /identity/v3/role_assignments?group.id=fc1fab04a1c24ad1bb4dfe03e1acb488&scope.project.id=8d062649850c4cc280349e14d6730b23&role.id=e5f3830b373b402693826265019b9344 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:35.267702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234e143-05b6-43e0-a176-d9557bd12fe1 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.293862 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.313133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.315261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.315261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.315261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.334041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:35.342118 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:35.343239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:35.344218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 165/325] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:35.351196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.384446 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.408935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.455645 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ad54f-9095-47c2-844d-5ff6f5ff276f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.498123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6f7a7-bc4a-408a-9bc5-674ef839e42d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.510100 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.511067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.511387 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.511829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.526173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:35.531687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:35.532922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:35.534268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 161/326] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:35.536046 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.536660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.536943 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.537213 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.537919 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:35.538221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:35.539080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.539698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:35.549768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 166/327] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:35.555629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.656489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de22ca-da7e-41fb-9d21-12bfd326d381 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.665959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.669844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e04f2-f83e-4e0c-9745-d2157ac1d0cc None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.671853 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.698049 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.698715 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.700079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.700353 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.701020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:35.701320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:35.702788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:35.708063 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.708693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.708977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.709255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.713114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 162/328] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:35.720270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.799798 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:35.802532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:35.855221 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef25ef-0510-4043-919c-5f6a54bd8584 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:35.899329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:35.900502 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:35.901294 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:35.901832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:35.917644 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.923589 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 167/329] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 06:00:35.928792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:35.963479 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:35.994150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:35.995896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.036094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41a7c2-4841-46cb-b6a4-0742e03b91c6 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:36.070114 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:36.070750 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:36.071035 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:36.071335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] PATH_INFO: `/v3/projects/eddd1a08322a42ffb5981fd594ba15aa` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:36.072477 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] RBAC: Authorizing `identity:get_project(project_id=eddd1a08322a42ffb5981fd594ba15aa)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:36.073913 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.075582 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 168/330] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/projects/eddd1a08322a42ffb5981fd594ba15aa => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:36.081588 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:36.097110 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 163/331] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 06:00:36.102609 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:36.148669 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:36.177612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:36.183473 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd967105-ac83-4556-a7ba-9b42a2babc3d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:36.202344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdcdc5-7a5a-4d9b-a721-969de62293a0 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:36.217486 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:36.218127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:36.218421 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:36.218702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] PATH_INFO: `/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:36.219884 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] RBAC: Authorizing `identity:get_group(group_id=fc1fab04a1c24ad1bb4dfe03e1acb488)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:36.221319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.223076 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 169/332] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 06:00:36 2022] GET /identity/v3/groups/fc1fab04a1c24ad1bb4dfe03e1acb488 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:36.228896 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:36.236129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:36.236762 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:36.237040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:36.237313 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] PATH_INFO: `/v3/projects/67b5d25652604b029091af61167fe6bf` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:36.238762 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] RBAC: Authorizing `identity:get_project(project_id=67b5d25652604b029091af61167fe6bf)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:36.240206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.242176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 164/333] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:00:36 2022] GET /identity/v3/projects/67b5d25652604b029091af61167fe6bf => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:36.247462 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:36.330172 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece335a6-70e7-434a-bc22-77536bb2933b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:36.346892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a06d-29af-4633-979e-bf281f1a5cee None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:36.364286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:36.364888 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:36.365522 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:36.365804 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:36.366555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] RBAC: Adding query filter params (group.id=fc1fab04a1c24ad1bb4dfe03e1acb488, role.id=e5f3830b373b402693826265019b9344, scope.project.id=eddd1a08322a42ffb5981fd594ba15aa) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:36.366863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:36.368336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.370053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:36.374865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 170/334] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 06:00:36 2022] GET /identity/v3/role_assignments?group.id=fc1fab04a1c24ad1bb4dfe03e1acb488&scope.project.id=eddd1a08322a42ffb5981fd594ba15aa&role.id=e5f3830b373b402693826265019b9344 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:36.381972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa71be7-092d-493c-9550-e4f128fcd4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:36.382638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71be7-092d-493c-9550-e4f128fcd4da None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:36.382940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71be7-092d-493c-9550-e4f128fcd4da None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:36.383206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71be7-092d-493c-9550-e4f128fcd4da None admin] PATH_INFO: `/v3/groups/a902199f76d54ab5981cb65eec0e68e6` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:36.384433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71be7-092d-493c-9550-e4f128fcd4da None admin] RBAC: Authorizing `identity:get_group(group_id=a902199f76d54ab5981cb65eec0e68e6)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:36.385929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71be7-092d-493c-9550-e4f128fcd4da None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.387676 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 165/335] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Oct 17 06:00:36 2022] GET /identity/v3/groups/a902199f76d54ab5981cb65eec0e68e6 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:36.393710 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:36.503487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e3c12-adc0-4b1c-aedf-4027440469f8 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:36.539272 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:36.539923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:36.540208 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:36.540482 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:36.541225 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] RBAC: Adding query filter params (group.id=a902199f76d54ab5981cb65eec0e68e6, role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=67b5d25652604b029091af61167fe6bf) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:36.541535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:36.542988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68180546-4d1f-463a-97fc-fa94f9cf973a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:36.549144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 171/336] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Oct 17 06:00:36 2022] GET /identity/v3/role_assignments?group.id=a902199f76d54ab5981cb65eec0e68e6&scope.project.id=67b5d25652604b029091af61167fe6bf&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:39.130388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192079d-91b0-4cf9-885c-2a009857d7d8 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.131462 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192079d-91b0-4cf9-885c-2a009857d7d8 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.131971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192079d-91b0-4cf9-885c-2a009857d7d8 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.133837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 166/337] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:39.139499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d874b-4c3c-42e6-99e1-91c488f1b3af None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.140319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d874b-4c3c-42e6-99e1-91c488f1b3af None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.140808 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d874b-4c3c-42e6-99e1-91c488f1b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.168577 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-1d2d874b-4c3c-42e6-99e1-91c488f1b3af None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:39.170269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-1d2d874b-4c3c-42e6-99e1-91c488f1b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:39.246776 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:39.257577 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 172/338] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:39.339387 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef333c-58f3-4a64-83fe-0842823309ab None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.339805 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef333c-58f3-4a64-83fe-0842823309ab None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.339924 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef333c-58f3-4a64-83fe-0842823309ab None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.341289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 167/339] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:39.346892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.347185 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.347487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.372263 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:39.373589 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:39.469937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 173/340] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:39.651469 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef333c-58f3-4a64-83fe-0842823309ab None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:39.669340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:39.753803 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:39.799189 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef333c-58f3-4a64-83fe-0842823309ab None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:39.861529 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f175b44-a536-4f72-a2aa-b670f3618b0e None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:39.863206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:39.864133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.864632 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.865136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.889948 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:39.891094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:39.893328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:39.911977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:39.912678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.912959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.913223 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:39.919260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:39.921014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:39.922008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:39.922008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:39.922008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:39.922008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 168/341] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:39.933232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:39.934715 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:39.936351 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-9228946e-81da-4592-9c60-4e0ec24494a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:39.966333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:39.967511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:39.968518 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:39.968518 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:39.968518 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:39.970485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 174/342] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:39.971953 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32271f73-be0d-48b7-b3dc-8d77c88b06d2 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.972465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32271f73-be0d-48b7-b3dc-8d77c88b06d2 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.972965 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32271f73-be0d-48b7-b3dc-8d77c88b06d2 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:39.974420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 175/343] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:39.979687 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:39.980216 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:39.980846 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.001899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:40.003091 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:40.018232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:40.056980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48807307-a6df-430c-91c4-3fde1e9d5a3a None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.082066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 176/344] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:40.088188 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.096477 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.097096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.097377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.097651 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.098294 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:40.098598 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.100218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4a32a-ada1-4894-84a3-73b5054804a3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.119226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 169/345] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:40.119226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612817d3-a5b8-4a67-9986-46fdf51578f4 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.119226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612817d3-a5b8-4a67-9986-46fdf51578f4 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.119226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612817d3-a5b8-4a67-9986-46fdf51578f4 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.119226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 170/346] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:40.120206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612817d3-a5b8-4a67-9986-46fdf51578f4 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.210637 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0720b2-60b3-436c-a04a-3f45a0b9e9df None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.214548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:40.249583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.250526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.251007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.251517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.252565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:40.253095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.253695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612817d3-a5b8-4a67-9986-46fdf51578f4 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.255222 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8e52e-bfaf-476d-8ffa-7abb1755008c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.268154 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 177/347] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:40.269814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.270342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.270838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.282143 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:40.290413 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:40.305061 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:40.305198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.305198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.305198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.305198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.305515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.305515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.366198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 178/348] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:40.367150 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.383453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 171/349] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:40 2022] POST /identity/v3/users => generated 246 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:40.385012 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.516184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:40.542980 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd42af-2b50-47cc-9457-d53d5ddfbed0 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.566713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.567376 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.567659 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.567931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.568592 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.570053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.595326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc12a7-05d9-4813-893f-b3f600773267 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.635048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.635761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.636043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.636319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.649695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.651188 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:40.669860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:40.672826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:40.673769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:40.673769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:40.673769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:40.673769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 172/350] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:40.673769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.709702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:40.725203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 179/351] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:40 2022] POST /identity/v3/users => generated 248 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:40.729406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.802950 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b2670-482b-4d93-b2c6-c2b681ffaeea None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.845061 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.846209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.846209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.846408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.858892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.860577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.868048 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7994ace-d92e-4792-b6d9-9438a5d60f4c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f9b43367-ab6d-49e2-99ca-2bcbf74656cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:40.877940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:40.879195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:40.880340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:40.880340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:40.880340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:40.882328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 173/352] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:40.884045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bcab5-0835-448b-805d-aac37b10ada6 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.884529 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bcab5-0835-448b-805d-aac37b10ada6 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.884982 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bcab5-0835-448b-805d-aac37b10ada6 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.886560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 174/353] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:40.892209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bcab5-0835-448b-805d-aac37b10ada6 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:40.918128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:40.918766 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.919050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.919506 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.920032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:40.920897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:40.922498 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648566ec-fccd-4e6d-9a11-d177572f5ab2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:40.937037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 180/354] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:40.938464 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:40.938761 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:40.939023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:40.958755 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:40.960043 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:40.963021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:41.004916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:41.024812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bcab5-0835-448b-805d-aac37b10ada6 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:41.053765 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 181/355] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:41.054528 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:41.087967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:41.089535 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:41.089818 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:41.090096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:41.093548 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:41.093864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:41.096420 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:41.119995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 175/356] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:41.125080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:41.222900 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad117e-9879-40d0-95ef-af6d06a39cff None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:41.286705 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:41.287347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:41.287632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:41.287907 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:41.288559 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:41.290006 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:41.314290 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:41.320882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7071-8bca-40f4-bed6-5c1666ea1f6c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:41.363321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:41.363994 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:41.364273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:41.364597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:41.365252 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:41.366670 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:41.402489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 182/357] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:41 2022] POST /identity/v3/users => generated 248 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:41.403701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:41.427598 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:41.439442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 176/358] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:41 2022] POST /identity/v3/users => generated 249 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:41.566595 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c25aa-c72e-4044-8da3-2443d58ff0bc None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:41.612776 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:41.613920 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:41.614431 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:41.614910 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:41.615852 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:41.617703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:41.659658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.sql.core [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 06:00:41.659658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 06:00:41.659658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [parameters: {'id': '8cbddd9cd20540fc86eed951e3cb27cd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 06:00:41.659658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result.read() Oct 17 06:00:41.666792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:00:41.668130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application yield resource Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise self.value Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application session.commit() Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:00:41.669849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:00:41.670892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:00:41.672865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result.read() Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:00:41.681850 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [parameters: {'id': '8cbddd9cd20540fc86eed951e3cb27cd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:41.685080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 06:00:41.691484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:41.693777 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 183/359] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 06:00:41 2022] POST /identity/v3/roles => generated 144 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:41.693932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:41.800899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7350d2b-5b2c-4333-a156-89efc40f88cc None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:41.837588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c1941-3477-4147-a173-7c3798d979db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:41.838231 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c1941-3477-4147-a173-7c3798d979db None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:41.838514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c1941-3477-4147-a173-7c3798d979db None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:41.838775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c1941-3477-4147-a173-7c3798d979db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:41.856162 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c1941-3477-4147-a173-7c3798d979db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-fd9c1941-3477-4147-a173-7c3798d979db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:41.862708 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:41.863730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:41.864985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 177/360] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:00:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:41.869963 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:41.946194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:41.998706 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1099a3c-0a11-4845-9f9a-b4aa0b397d38 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:42.042633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:42.043256 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:42.043562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:42.043836 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:42.044507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:42.044812 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:42.046758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c8855-4e23-47ac-89d9-d3ae613ba8a6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:42.059355 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 184/361] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:42.897026 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb69ab-0f3c-45c7-be83-a298344f97ab None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:42.897580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb69ab-0f3c-45c7-be83-a298344f97ab None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:42.897580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb69ab-0f3c-45c7-be83-a298344f97ab None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:42.899269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 178/362] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:42.904332 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c9d7-929c-4c9e-8cbc-9c2d593b5ed7 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:42.905412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c9d7-929c-4c9e-8cbc-9c2d593b5ed7 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:42.905501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c9d7-929c-4c9e-8cbc-9c2d593b5ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:42.928264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-a025c9d7-929c-4c9e-8cbc-9c2d593b5ed7 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:42.929470 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-a025c9d7-929c-4c9e-8cbc-9c2d593b5ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:43.006666 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 185/363] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:43.218287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b47dd-b12d-4769-9703-4447a0087762 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:43.218477 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b47dd-b12d-4769-9703-4447a0087762 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:43.218754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b47dd-b12d-4769-9703-4447a0087762 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:43.220399 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 179/364] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:43.225569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:43.225856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:43.226126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:43.250259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:43.251473 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:43.317901 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:43.348763 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 186/365] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:43.370213 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b47dd-b12d-4769-9703-4447a0087762 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:43.457820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:43.511858 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b47dd-b12d-4769-9703-4447a0087762 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:43.557344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:43.557986 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:43.558268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:43.558543 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:43.572750 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:43.579745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:43.583588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:43.584474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 180/366] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:43.588416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:43.636401 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:43.779331 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11a4b8-e6c7-4ea3-b03f-549ddd8fbae0 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:43.848538 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a7204-2532-4c50-915b-7731c2bf74b9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:43.858568 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:43.859801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:43.859801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:43.859801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:43.860314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:43.863445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:43.863445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:43.878565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 187/367] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:43.882735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:43.889622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:43.923035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:43.924479 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:43.924954 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:43.925427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:43.944215 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:43.955324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:43.955798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:43.956743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:43.957625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:43.961023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 181/368] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:43.964855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.041006 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc28e70-c6e5-4cc4-9c35-e24825874568 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.095709 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.096370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.096650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.096923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.122459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.124526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.150753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bba684-e5ad-4e34-80e3-e32b0c064411 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:44.154615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:44.156889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:44.159072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:44.159072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:44.159072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:44.159072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 188/369] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:44.161958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.193778 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.194445 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.194729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.195003 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.195697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:44.196001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.197386 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.216158 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 182/370] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:44.222481 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.285615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:44.318287 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:44.326119 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd01e-be2a-49c8-85d6-21fe7f294890 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.350394 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089676a-3cfb-434f-8127-3ee0bcb24110 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.369138 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.369751 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.370027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.370291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.370925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:44.371219 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.372608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab861c-48b5-452f-a772-4eec1eacbf45 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.384132 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 189/371] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:44.385380 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52269c55-4cf8-4845-910c-5531d0a9c818 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.385660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52269c55-4cf8-4845-910c-5531d0a9c818 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.385938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52269c55-4cf8-4845-910c-5531d0a9c818 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.387752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 190/372] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:44.388451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ffd21-7163-427c-80b2-54274d5b42ee None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.389507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ffd21-7163-427c-80b2-54274d5b42ee None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.389782 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ffd21-7163-427c-80b2-54274d5b42ee None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.390959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 191/373] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:44.391675 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ffd21-7163-427c-80b2-54274d5b42ee None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.396416 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.397309 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.397819 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.398319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.410545 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.412343 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f678c846-76a9-47fc-b511-251906e055f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:44.428643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:44.429677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:44.430606 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:44.430606 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:44.430606 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:44.432483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 183/374] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:44.433660 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab9403-f6ba-4be1-8ca0-79b62f68f1a5 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.433946 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab9403-f6ba-4be1-8ca0-79b62f68f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.434215 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab9403-f6ba-4be1-8ca0-79b62f68f1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.452477 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-71ab9403-f6ba-4be1-8ca0-79b62f68f1a5 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:44.453660 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-71ab9403-f6ba-4be1-8ca0-79b62f68f1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:44.510553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ffd21-7163-427c-80b2-54274d5b42ee None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.523060 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 184/375] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:44.524390 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.524676 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.524952 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.552050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.552747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.553029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.553341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.554513 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:44.556027 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:44.562439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.564001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.577114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:44.579554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:44.580634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:44.581635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 192/376] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:44.586335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.605731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:44.636100 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 185/377] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:44.637309 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.757586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f23dc-bc53-46df-8e79-0f1c2205a6f0 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.771337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc2924-46c0-43a0-bf85-c2d181faba7b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.813920 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.814534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.814804 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.815074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.815830 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:44.816131 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.816976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.817563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.817682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.818009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.818344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:44.819055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:44.819493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:44.820860 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:44.832676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 193/378] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:44.833593 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.858817 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 186/379] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:44.859857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:44.875736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:44.920789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:44.955743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddcb3c-7369-4a18-b883-5c82c9231f05 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.979334 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87959cdf-77eb-4f83-918c-624365b6c2a7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:44.995922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:44.996562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:44.996841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:44.997108 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.008233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:45.013645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.015206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:45.016125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 194/380] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.016876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.026028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.026680 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.026964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.027238 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.036377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.037886 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:45.052652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.053679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 187/381] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.055566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.127240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.128659 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263031b2-ef36-4855-973a-92f96d47f27f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.157949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1318-dd36-4f75-bee5-f0cc928d59d4 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.162609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.163244 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.163555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.163830 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.166469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.174489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.175978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:45.190164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.191151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:45.192142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:45.192142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.192142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:45.192320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.192415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 195/382] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.192494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.192494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.192494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.192702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.207035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:45.212398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.213533 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 188/383] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.214408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.294967 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25d26a-4f7e-4c12-abb8-e43959d08ff1 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.314279 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae3475-8403-48aa-bbc4-d27f9239b470 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.328835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.329437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.329714 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.329986 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.330645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:45.330946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.332354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3caa23-c596-4c18-bccb-bdb4bccbee5d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.340132 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 196/384] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:45.341264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc071-3e19-4865-847c-75efb5c0926c None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.341546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc071-3e19-4865-847c-75efb5c0926c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.341824 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc071-3e19-4865-847c-75efb5c0926c None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.343606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 197/385] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:45.344297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc071-3e19-4865-847c-75efb5c0926c None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.346035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.351504 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.352311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.352592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.352865 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.353593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:45.353899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.355350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.372606 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 189/386] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:45.373780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.401469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.452026 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc071-3e19-4865-847c-75efb5c0926c None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.474224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46efa-bd2a-41c9-a68f-10234cec49d3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.485719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.486324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.486601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.486868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.487529 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:45.487826 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.489191 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.501229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 198/387] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:45.504171 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.509432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.510040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.510322 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.510590 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.511251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:45.511588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.512922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d943f-f819-44a7-a06f-789a9c9ce8b5 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.521259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 190/388] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:45.522923 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.523208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.523507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.541348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:45.542568 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:45.588438 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.602261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44369a-0320-42a6-8102-ad717e4fda57 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.608908 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 191/389] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:45.609715 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.622147 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.646220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.647144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.647144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.647473 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.658593 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.660011 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:45.676203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.677229 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:45.678177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:45.678177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.678177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:45.678177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 199/390] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.678177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.716664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4643c75-1372-4e7e-adbd-59e09122e727 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.751016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.751682 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.751962 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.752236 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.762257 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.764317 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:45.780578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.781576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:45.782555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:45.782555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.782555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:45.782555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 192/391] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.782555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.807422 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90aefa-4014-44c2-bebe-9e2d6c9759b1 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.819418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.844098 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.844739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.845022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.845296 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.852184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.853636 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:45.865489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.866430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 200/392] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.867387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.892941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:45.893358 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd5dd8-dcdd-46c9-9de0-a06dac7aa108 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:45.928122 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:45.928688 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:45.928971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:45.929246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:45.939428 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:45.940884 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:45.956735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:45.957873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:45.958813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:45.958813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:45.958813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:45.958813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 193/393] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:45.958813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:45.974691 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab75861-c7e3-49ae-9e5e-4f844784310c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.009017 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.009648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.009925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.010194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.010828 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:46.011129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.012527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.024122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 201/394] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:46.024744 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.046734 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.061474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383294fe-a696-4827-ad28-e1372f35946d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.094966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.095599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.095880 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.096165 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.096782 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:46.097434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.098795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.104386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.110562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 194/395] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:46.111518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.125290 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdeb12-442a-43aa-9ccb-930ac95ad46c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.159099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.159736 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.160018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.160298 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.160986 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:46.161292 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.162727 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.180352 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 202/396] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:46.181198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.218731 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bf1f0-48b4-4b2a-9093-8fee0c5d9ad2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.260263 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.260899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.261176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.261449 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.274209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.275971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.284042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:46.292400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:46.293574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:46.294722 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:46.294722 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:46.294722 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:46.300700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 195/397] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:46.300700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.305070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc11f0-5805-4eb6-a537-938ad68c5c81 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.342935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.343646 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.343932 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.344204 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.344856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:46.345164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.346555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.360356 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 203/398] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:46.361209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.363915 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.419270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc031d9-567a-4721-816b-f61833ae1e11 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.458118 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.459059 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.459366 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.459642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.473005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.474033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337cbc4-d595-4494-bef3-f8066302f1bc None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.474456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:46.486622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:46.488021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 196/399] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:46.489677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.513022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.513720 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.514540 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.514817 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.521801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.523247 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:46.534542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:46.535512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 204/400] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:46.536465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.553509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.594994 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d730f83-c00f-4e2a-9ddc-21a901187954 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.597609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.629849 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.630521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.630803 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.631080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/9dcc0e6649de439eb130a3b5f5bc1b91/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.637865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc7e1-2292-48ba-9e84-bf1ee29903ed None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.646514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=9dcc0e6649de439eb130a3b5f5bc1b91, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.647997 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.671921 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.672524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.672806 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.673082 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.673723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:46.674023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.675406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.687687 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 205/401] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:46.688471 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.734187 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 197/402] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:46 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/9dcc0e6649de439eb130a3b5f5bc1b91/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:46.734938 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.802515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b09354-7b53-4878-986f-3c1d6704bd03 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.830130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.842643 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.843306 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.843604 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.844023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.848198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2b9a7-12b2-450c-879b-3467c549200e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:46.851955 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.853416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.859512 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:46.867326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:46.874049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 206/403] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:46.875345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.895117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:46.895782 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:46.896063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:46.896334 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:46.897073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:46.897385 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:46.899107 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:46.918111 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 198/404] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:46.919037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:46.992307 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641ce4-f094-4e7f-b684-16e8570f37ad None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.028528 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378543f-3be1-41f1-900b-4834fe2256c8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.030096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.030823 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.031226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.031535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.032235 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:47.032542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.034004 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.051713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 207/405] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:47.052597 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.066636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.067277 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.067562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.067826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.068473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.069877 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.098494 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:47.159335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 199/406] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:46 2022] POST /identity/v3/users => generated 247 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:47.159335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.163142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2367573-78b7-4c2f-bd9d-c02eb9da3505 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.183600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:47.217820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.218472 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.218755 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.219023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.219883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:47.220201 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.221788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.275724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 208/407] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:47.276371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.277785 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfe2db-6f1b-485b-a215-bc20fe093c17 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.326748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.327842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.328347 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.328819 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.343558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.344507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:47.362274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:47.363329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:47.364816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:47.364816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:47.364816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:47.364816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 200/408] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:47.365072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.415946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:47.418885 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719f5a1-1b61-44dd-8ff9-f9f62af6ee69 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.457050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.457715 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.457994 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.458269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/4f09eb0f455247afb8de8370261178d8/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.462130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:47.480806 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=4f09eb0f455247afb8de8370261178d8, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.482174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.493649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b5297-27fd-48a2-a03e-695dc979e13b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.530128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.530785 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.531066 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.531421 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.542900 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.544379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.549149 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 209/409] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:47 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/4f09eb0f455247afb8de8370261178d8/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:47.550311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:47.562370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:47.563497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:47.564932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:47.564932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:47.564932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:47.564932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 201/410] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:47.569759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.694832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e2856-74ec-4cef-aed7-ad4c6bd7240a None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.709896 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf508c-8382-4d8e-90d2-e22b0c01a620 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.724809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:47.734661 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.738630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.738630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.738630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.738630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:47.738630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.738630 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.752063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 202/411] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:47.757767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.760799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.762569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.763065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.763551 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.764436 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:47.764923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.766537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:47.784294 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 210/412] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:47.792389 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:47.819237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:47.932704 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad2b45-318f-4e0c-9e15-b263c06602f7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.940241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8cc8-3081-4cba-a845-fec7af352a5c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:47.984162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:47.984895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:47.985261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:47.985602 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:47.992741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:47.994300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:48.007090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:48.008071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:48.009386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 211/413] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:48.012034 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:48.012690 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:48.012972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:48.013244 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:48.014493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:48.028435 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:48.029937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:48.053893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:48.054855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:48.055829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:48.063742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 203/414] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:48.070291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:48.105999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:48.166244 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:48.173645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583132ef-9535-4c07-82a5-59edf09fd3f5 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:48.243111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:48.243906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:48.244282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:48.244650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:48.245469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:48.245889 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:48.247457 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:48.268434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a60ae-14ff-4a70-9ea9-3843df7d87ea None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:48.274296 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 212/415] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:48.299051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:48.335250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:48.336473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:48.337271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:48.338013 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:48.338877 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:48.339514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:48.341890 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:48.377752 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 204/416] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:48.384448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:48.481886 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246fb07-847c-4096-a055-db5affdb34a9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:48.543709 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:48.549738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ff659-ac98-4c00-b84a-eefbbeac9c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:48.550419 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ff659-ac98-4c00-b84a-eefbbeac9c0f None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:48.550704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ff659-ac98-4c00-b84a-eefbbeac9c0f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:48.550980 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ff659-ac98-4c00-b84a-eefbbeac9c0f None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/7a33a651f55d4e43985780777a4b2003/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:48.570768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf6ad5-9cb5-490c-943b-91e4e31097f1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:48.584237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ff659-ac98-4c00-b84a-eefbbeac9c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=7a33a651f55d4e43985780777a4b2003, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:48.585818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ff659-ac98-4c00-b84a-eefbbeac9c0f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:48.587936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:48.621949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0541903-0c82-4af7-b47f-86ff415ab9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:48.622922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541903-0c82-4af7-b47f-86ff415ab9e6 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:48.624063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541903-0c82-4af7-b47f-86ff415ab9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:48.624513 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0541903-0c82-4af7-b47f-86ff415ab9e6 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/0a76012127184a19bd9ccab445d24d75/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:48.640724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0541903-0c82-4af7-b47f-86ff415ab9e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=0a76012127184a19bd9ccab445d24d75, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:48.642668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0541903-0c82-4af7-b47f-86ff415ab9e6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:48.656829 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 213/417] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:48 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/7a33a651f55d4e43985780777a4b2003/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:00:48.720350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 205/418] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:48 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/0a76012127184a19bd9ccab445d24d75/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:49.247320 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393cb8-0b78-4e27-859f-f4f94fd3162b None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:49.247848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393cb8-0b78-4e27-859f-f4f94fd3162b None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:49.247848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393cb8-0b78-4e27-859f-f4f94fd3162b None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:49.249509 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 214/419] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:49.254732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9a4c-2e2a-4e73-9a0b-4d047cd1069d None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:49.255139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9a4c-2e2a-4e73-9a0b-4d047cd1069d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:49.255336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9a4c-2e2a-4e73-9a0b-4d047cd1069d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:49.284311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-2bea9a4c-2e2a-4e73-9a0b-4d047cd1069d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:49.285530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-2bea9a4c-2e2a-4e73-9a0b-4d047cd1069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:49.374674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 206/420] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:49.680971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc576-e57b-4fbf-9a09-346a9643aa97 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:49.681633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc576-e57b-4fbf-9a09-346a9643aa97 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:49.681824 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc576-e57b-4fbf-9a09-346a9643aa97 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:49.683612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 215/421] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:49.696320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:49.696863 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:49.697355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:49.724282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:49.725855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:49.774938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc576-e57b-4fbf-9a09-346a9643aa97 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:49.800566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 207/422] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:49.913869 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc576-e57b-4fbf-9a09-346a9643aa97 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:49.950872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:49.951921 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:49.952431 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:49.952923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:49.964390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:49.970259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:49.971232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:49.972140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:49.974835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 216/423] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:49.980271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.000325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.095018 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621fb5d-4fe9-4c81-987b-b8436119d5b1 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.133346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.134399 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.134939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.135453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.136408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:50.137585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.138257 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.139728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:50.155135 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 208/424] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:50.160246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.217235 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.288397 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961f8d4-695f-419f-9473-da27228cefbf None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.307686 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.318837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8755e-8693-488f-a24f-dceda4986222 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.328692 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dd3fd-bd32-437f-9759-6eca12738152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.329904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dd3fd-bd32-437f-9759-6eca12738152 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.330270 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dd3fd-bd32-437f-9759-6eca12738152 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.330789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dd3fd-bd32-437f-9759-6eca12738152 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.343516 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dd3fd-bd32-437f-9759-6eca12738152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-781dd3fd-bd32-437f-9759-6eca12738152 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:50.351443 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:50.352423 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:50.353370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:50.356673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 217/425] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:50.358757 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2509e9-8fc9-4311-addc-27b2b4098531 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.359237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.359523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2509e9-8fc9-4311-addc-27b2b4098531 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.359922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.360201 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.360476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.360687 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2509e9-8fc9-4311-addc-27b2b4098531 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.362773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 218/426] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:50.363899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2509e9-8fc9-4311-addc-27b2b4098531 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.378347 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.380169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f0ebc5a6-c208-44ab-a2c1-1d3144886414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:50.400169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:50.401153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:50.402092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:50.402092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:50.402092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:50.402092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 209/427] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:50.402979 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.403266 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.403580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.428966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:50.430587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:50.477754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.481529 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2509e9-8fc9-4311-addc-27b2b4098531 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.518815 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.519487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.519892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.520175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.520852 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:50.521160 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.522551 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:50.523139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 210/428] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:50.524244 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.543058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 219/429] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:50.548318 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.593090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.703575 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bb7fa-f842-4f2f-91ba-a9a309161e74 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.705181 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9141503-6f0e-4142-9df4-b96c61d8e2d2 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.743041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.744045 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.744559 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.745062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.757202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.759057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:50.763142 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.763830 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.764113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.764390 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.765041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:50.765345 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.766713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:50.778371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 211/430] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:50.788240 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:50.789183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:50.790166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:50.791112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:50.791112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:50.791112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:50.793334 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 220/431] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:50.799347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.825516 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.911833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae04ab-1727-4b03-96e4-b3cc8ffa602e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.936672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:50.940320 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97eb53-6393-49e3-b6df-36cc6c55e823 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:50.952174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.952808 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.953090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.953740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.962808 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.964458 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:50.978747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:50.979836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:00:50.980845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:50.983850 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 212/432] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:50.984586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:50.989173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:50.991368 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:50.991879 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:50.992383 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:50.993308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:50.993835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:50.995489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.014461 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 221/433] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:51.015673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.109857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db34085-8c84-4b58-90f1-87c7089dac17 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.124939 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.155631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.156180 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.156496 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.156716 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.169152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b6e16-c85b-4e5b-baf1-0513e0bc60a1 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.169542 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:51.176038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:51.177036 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:51.178011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:51.183042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 213/434] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:51.183918 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.220765 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.221923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.222364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.222801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.224239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:51.225899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.228122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.234060 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.253512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 222/435] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:51.254581 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.302972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdfc98-7de9-4be7-97c2-bb072f83903c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.341698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.342337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.342707 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.343052 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.351939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.353399 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1e225366-d138-4d9b-b36d-ab7ec537c897 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:51.368763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:51.369942 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:51.370939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 214/436] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:51.371626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7465c75-c0bc-4a01-965d-8f3eb4b922c2 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.371824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7465c75-c0bc-4a01-965d-8f3eb4b922c2 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.372016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7465c75-c0bc-4a01-965d-8f3eb4b922c2 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.377605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 215/437] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:51.377605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7465c75-c0bc-4a01-965d-8f3eb4b922c2 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.377853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836e61-a468-4b78-b445-9b5357297b5d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.408246 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.413351 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.413970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.414253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.414526 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.415198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:51.415525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.417651 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.430447 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 223/438] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:51.430447 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.485448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.489289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7465c75-c0bc-4a01-965d-8f3eb4b922c2 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.525243 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.525772 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.526047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.526313 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.533545 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd92be9-0f7a-4b9c-8e76-36b56fdab762 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.568071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.568811 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.569192 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.569571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.570456 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:51.570871 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.572456 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70556a55-5d30-4c94-8f0d-6bfa1da755c5 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.590280 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 224/439] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:51.591540 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988102c8-70fe-45b5-bcdb-65bbcb8bf593 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.591834 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988102c8-70fe-45b5-bcdb-65bbcb8bf593 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.592115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988102c8-70fe-45b5-bcdb-65bbcb8bf593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.593441 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.595067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.609492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-988102c8-70fe-45b5-bcdb-65bbcb8bf593 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:51.610710 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-988102c8-70fe-45b5-bcdb-65bbcb8bf593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:51.658313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.677960 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 225/440] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:51.679189 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01716b-8274-472f-9f58-59f3bcecda85 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.679514 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01716b-8274-472f-9f58-59f3bcecda85 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.679798 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01716b-8274-472f-9f58-59f3bcecda85 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.681031 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 226/441] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:51.681994 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01716b-8274-472f-9f58-59f3bcecda85 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.703948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.707665 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 216/442] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:51.712455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.808190 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01716b-8274-472f-9f58-59f3bcecda85 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.812057 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72836ea9-2349-482e-b2c5-59ffa299a034 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:51.841947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.842708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.843068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.843373 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.847160 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:51.847988 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.848436 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.848856 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.853357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.854801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.860906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:51.862619 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-e5000c3f-9a5d-44d6-b95c-90d1064f2d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:51.869904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:51.871037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:51.872117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:51.872117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:51.872117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:51.872117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 227/443] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:51.877295 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:51.877586 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:51.877863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:51.880629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:51.881740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:51.882842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:51.882842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:51.882842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:51.884349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 217/444] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:51.885418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:51.900089 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:51.901292 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:51.928633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.947768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:51.968975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 228/445] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:51.969814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.000801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29098a-6067-4625-8892-aa91b83321d3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.039291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.040350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.040700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.041047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.052329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.053777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:52.069207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:52.070192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:52.071635 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:52.071635 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:52.071635 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:52.071838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ac0d-68b0-4166-a860-b15672af847a None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.075053 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 218/446] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:52.076443 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.105713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.106331 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.106608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.106877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.108097 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:52.108399 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.109770 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.121273 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 229/447] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:52.122139 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.147368 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.192532 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597e834-b175-45d7-96ed-d39aff741f1d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.196078 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.224469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83cecf-5d18-4f31-a5b3-9b8a236defb6 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.229939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.230551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.230830 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.231098 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.231768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:52.232459 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.233531 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.245614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 219/448] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:52.246476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.261890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.262529 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.262813 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.263092 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.273918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:52.278838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:52.280073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 230/449] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:52.281112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.349085 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2298172-0022-483f-bee7-d2f87631b258 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.363703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.379233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53126dd4-ae8f-4c2c-87c7-784ce0be160c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.382614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.383248 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.383557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.383831 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.384463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:52.384766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.386130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.397476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 220/450] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:52.398472 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.413564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.414212 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.414487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.414762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] PATH_INFO: `/v3/users/glance` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.422010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.423475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.424641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:52.435843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:52.439100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 231/451] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:52.440673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.506045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab041a-8bc4-4a30-a971-de606c1e3788 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.538241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868049d-3365-4025-b1f0-57c08381c75f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.538693 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.539355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.539638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.539908 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.550521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:52.555851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:52.557259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:52.558418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:52.562611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 221/452] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:52.563415 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.572460 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.573173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.573546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.573894 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.580796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.582430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:52.594300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:52.595479 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 232/453] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:52.596622 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.603984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.651390 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.667035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7642e-491f-46bd-915a-4d5363a2681b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.702320 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a207-de50-49c1-b5dc-80bec3364b23 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.705080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.705686 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.705960 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.706229 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.706880 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:52.707178 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.708559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.716244 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 222/454] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:52.717068 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.739652 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.740281 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.740561 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.740833 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.747404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.748933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:52.761336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:52.762267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 233/455] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:52.764135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.818267 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a08822-1d92-4a69-905b-d2937d4ba1bc None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.825085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.851055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.851690 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.851974 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.852251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.852999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:52.853305 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.854782 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.862983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d9d5d-24ed-4686-8f88-d19187def967 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:52.872110 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 223/456] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:52.872995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.876550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:52.899434 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:52.900042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:52.900327 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:52.900601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:52.901266 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:52.901571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:52.903134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:52.911693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 234/457] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:52.912511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:52.973856 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e4dfa-d0b5-4759-903a-91d4396408c3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.008086 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.008694 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.008995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.009266 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.009961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:53.010266 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.011735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.013301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41577a25-eeda-4073-aee1-9044b1ea63c9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.022771 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 224/458] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:53.023587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.038775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.047182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.047844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.050787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.050787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.059270 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.062078 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:53.077073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:53.078177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:53.079111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:53.079111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:53.079111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:53.079111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 235/459] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:53.079111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.095408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.133011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1790d-f617-4719-a480-9bf9ee437891 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.166235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.166840 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.167125 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.167430 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.168115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:53.168424 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.169968 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.175737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cc452-bab7-4990-a8f2-451ddd6ade67 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.181892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 225/460] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:53.182705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.209775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.210379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.210661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.210939 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.256499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.278416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.280464 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.288810 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aa9d7-abeb-462a-af8e-c8d10feca9b8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.312734 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.324096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.324732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.325013 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.325289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.336172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.337633 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:53.351964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:53.353340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:53.354377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:53.354377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:53.354377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:53.354377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 226/461] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:53.354377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.382291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 236/462] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:53.383138 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.454710 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bfff0-ca72-441a-bfc7-9695a7538706 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.457365 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.483124 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd0575-0459-4fc6-9a9e-e7bb53914548 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.489382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.490043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.490320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.490592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/1e93da8c826f427a8a7fb357f6dcfc67/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.504900 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=1e93da8c826f427a8a7fb357f6dcfc67, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.506350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.517360 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.517965 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.518249 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.518524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.519174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:53.519515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.520882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.532486 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 237/463] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:53.533269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.619723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 227/464] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:53 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/1e93da8c826f427a8a7fb357f6dcfc67/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:53.630046 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b4cb9-2bd0-456e-8262-431e4df1cf0f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.636407 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.667724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.694328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.695613 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.696165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.696536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.697833 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=9dcc0e6649de439eb130a3b5f5bc1b91) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:53.698364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.701574 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.723333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 238/465] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/role_assignments?user.id=9dcc0e6649de439eb130a3b5f5bc1b91&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:53.723333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.723333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.786864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f9b46-7096-4798-bbc7-b1fc1f7b79b9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.836951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.837595 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.837874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.838148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.839115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:53.839507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.840921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.851638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc63b40-8781-45bf-a515-bf639e910287 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:53.856926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 228/466] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:53.858016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.901060 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:53.901714 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:53.902111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:53.902446 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:53.923069 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:53.924571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:53.941949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:53.942983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:53.943974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:53.943974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:53.943974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:53.943974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 239/467] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:53.943974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:53.968212 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:53.986837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec031-79bb-481a-bd64-aba34494d924 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.018460 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.030586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.031421 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.031795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.032162 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.040912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.042127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:54.054605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:54.055822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:00:54.057269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:54.059004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 229/468] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:54.060046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.084707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97a2e8-2e6d-497c-ba84-9f90ba780996 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.137448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.138244 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.139178 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.139562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.149357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.151173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:54.163001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:54.164218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 06:00:54.165416 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:54.167156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 240/469] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:54.168166 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.183448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd0d0-0bb9-46dd-ad8e-e1ee8a86551e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.229524 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.231024 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.231501 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.232726 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.233531 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:54.234039 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.236456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.255624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 230/470] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:54.256753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.287682 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.289823 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf8312-82dd-407a-b7b0-f097c63181d8 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.309052 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.336753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.337615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.338337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.338793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.339998 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:54.340458 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.342838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.361944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02c4c3-4fd9-4823-9cfa-bfe4ac1ec9d4 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.365702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 241/471] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:54.373408 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.412072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.412965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.413824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.414270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.415455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:54.415914 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.418402 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.444239 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 231/472] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:54.445400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.485285 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b3c90-5d68-46c7-acd4-0edabb6fea4e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.518839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.523770 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.524750 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.525255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.525703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.534112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.536003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:54.551206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:54.553099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:54.554396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:54.562052 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 242/473] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:54.562052 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.579383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987c65-32da-4efa-8b5a-598e68f17fbf None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.600291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.629534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.631402 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.631832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.632496 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.676433 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d93b03-7edb-4135-907d-3c8adf5a5204 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:54.722519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:54.723708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:54.724090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.724355 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:54.724769 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:54.725832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.788009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.817415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:54.820003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:54.835723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 232/474] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:54.836767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.864811 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:54.943688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 243/475] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:54.944403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:54.968033 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56750ff-8670-43bc-89fd-fc4e8bf14136 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.015924 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.016915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.017398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.017866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.019033 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:55.020417 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.022065 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.042177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 233/476] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:55.042177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.055176 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70777ad-5037-4312-98a5-253451fe2dad None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.084103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:55.090996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:55.097344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.098037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.098321 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.098593 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.113086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.115031 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:55.136605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:55.137660 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:55.138650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:55.138650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:55.138650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:55.141511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 244/477] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:55.142311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.171905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ca5cb-981b-4c07-8a96-827d6b57fb24 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.215865 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.217553 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.218040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.218955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.231766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.233792 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:55.253809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:55.254843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:55.255847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:55.255847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:55.255847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:55.261112 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 234/478] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:55.261112 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.262956 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345643c-8b7d-4f36-ad79-735709aaf087 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.307516 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.308545 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.308838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.309115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.309902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=4f09eb0f455247afb8de8370261178d8) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:55.310761 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.312685 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.327326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 245/479] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/role_assignments?user.id=4f09eb0f455247afb8de8370261178d8&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:55.327326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.391983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:55.426810 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab073856-e218-4d15-abd8-3be06c889cdb None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.441440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:55.485201 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.487898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.488403 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.489714 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.490567 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:55.491436 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.498338 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.528998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 235/480] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:55.531451 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.576960 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b0d1-efd4-400d-a232-c45c127c6ee0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.615693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.616314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.616598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.616870 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.617537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:55.617841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.619220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.633030 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 246/481] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:55.644831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.654463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1678cb2-d130-405f-a7e8-f69f6ecda145 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.702693 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.703677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.704115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.704891 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.709691 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:55.714991 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.716710 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:55.731200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:55.732947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:55.734176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:55.736747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 236/482] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:55.742755 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.784496 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8a4fc-3598-48d4-92ec-5cc55ca31031 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.799921 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:55.849522 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.850164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.850450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.850728 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.865965 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.867472 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:55.881394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:55.882403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:55.883663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 247/483] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:55.894945 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:55.935875 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ea646-37fb-43bc-911c-b3fe9856cc0e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:55.974696 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:55.975352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.975638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.975914 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:55.976631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:55.976939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:55.978471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60639c1a-14fd-4a75-b01b-064f31cf3484 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:55.992494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 237/484] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:55.993704 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3cb83-9963-469d-8f77-567f638122e5 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:55.993990 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3cb83-9963-469d-8f77-567f638122e5 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:55.994264 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3cb83-9963-469d-8f77-567f638122e5 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.001287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 238/485] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:56.002056 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3cb83-9963-469d-8f77-567f638122e5 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.019608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6cf6d-b629-4034-9ced-b8cd4e560277 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:56.064373 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:56.065271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:56.066844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.067325 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.068071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.069297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:56.070034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:56.072249 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e02e-0c07-455a-83ee-f8875fbb314d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:56.089745 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 248/486] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:56.091969 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb6244-129b-4847-9770-7b8da75306e3 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.092666 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb6244-129b-4847-9770-7b8da75306e3 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.093674 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb6244-129b-4847-9770-7b8da75306e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.114319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3cb83-9963-469d-8f77-567f638122e5 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:56.116887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-59bb6244-129b-4847-9770-7b8da75306e3 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:56.118100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-59bb6244-129b-4847-9770-7b8da75306e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:56.143821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:56.150813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:56.154045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.154045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.154045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.154045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=7a33a651f55d4e43985780777a4b2003) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:56.154045 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:56.154581 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4d8a3-3901-46e2-bcbc-b1c739e932f2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:56.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 239/487] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/role_assignments?user.id=7a33a651f55d4e43985780777a4b2003&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:56.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd56a9-3361-45b0-ad22-c41ddd2ef791 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd56a9-3361-45b0-ad22-c41ddd2ef791 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd56a9-3361-45b0-ad22-c41ddd2ef791 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 240/488] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:56.167350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd56a9-3361-45b0-ad22-c41ddd2ef791 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.219470 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 249/489] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:56.220902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.221189 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.223517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.258448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:56.262054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:56.343175 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd56a9-3361-45b0-ad22-c41ddd2ef791 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:56.372427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:56.392217 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:56.392861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.393100 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.393388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.394115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=0a76012127184a19bd9ccab445d24d75) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:56.394428 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:56.396102 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:56.396795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 250/490] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:56.405055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 241/491] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/role_assignments?user.id=0a76012127184a19bd9ccab445d24d75&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:56.529937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.629388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:56.642678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04b1de-f16c-4fc5-944d-8d23b11d3b5f None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:56.678541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:56.679177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.679489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.679763 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.691164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:56.696041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:56.697055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:56.697996 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 251/492] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:56.703382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.718929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.825345 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b7d1-5254-4c07-97f7-b081c9616f77 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:56.865291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befb10f-59a7-4e15-bee6-068b544e6b52 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:56.881984 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:56.882614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.883068 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.883274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.883976 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:56.884283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:56.885649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:56.900106 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 242/493] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:56.906237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.918803 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:56.919480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:56.919799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:56.920071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:56.933865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:00:56.939499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:56.940536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:56.941466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 252/494] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:56.947608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:56.961445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:56.996578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:57.076690 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4b5bc-0458-4369-b3d6-5616ae8ce896 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.100949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a064a-1642-4a88-ba10-db2feb26ac9e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.141807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.142449 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.142730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.143001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.162698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.162698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.165909 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.166745 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.167185 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.167641 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.168468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:57.168930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.175677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:57.187588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:57.188616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:57.189662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:57.189662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:57.189662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:57.192025 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 253/496] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:57.192173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 243/496] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:57.197495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:57.198803 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:57.323169 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:57.374131 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60018576-a2ab-440c-86e9-577935694c91 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.393939 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:57.401727 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d9abf-da67-43e2-81c6-218d05ec1021 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.415938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.417251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.417539 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.417967 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.418539 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:57.419242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.421276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.437254 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 254/497] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:57.443050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:57.504833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.505623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.505905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.506181 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.537316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.539135 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.563185 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe266d-5d95-440d-8631-c7fc8661e79b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:57.568103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:57.569126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:57.570090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:57.570090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:57.570090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:57.571758 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 244/498] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:57.581038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:57.621856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:57.634415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.636151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.637267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.637546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.656087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.657556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:57.680136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:57.681221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:00:57.682542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:57.688111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 255/499] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:57.697650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:57.757914 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d2b5b-fac5-49b2-be8d-4fd2ba8965b1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.793615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.794234 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.794550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.794832 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.795509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:57.795820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.797274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.810146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 245/500] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:57.815550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:57.830969 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:57.925845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c407ff6-8812-4e85-a22b-84c9733b8a4f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.961499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0344516-eab1-4da7-a396-6b93f521e2cc None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:57.964566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:57.965216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:57.965546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:57.965823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] PATH_INFO: `/v3/users/nova` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:57.973431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:57.974892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:00:57.988381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:57.989467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:57.990620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 246/501] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:00:57.995501 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.035737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.036762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.037287 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.037776 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.038812 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:58.039883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.042030 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c72abf-7a78-48fc-b857-f1003be57405 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.062437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 256/502] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:58.063723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2742cf8-fdb1-4628-8e6f-6de0ea0e0b89 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.064075 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2742cf8-fdb1-4628-8e6f-6de0ea0e0b89 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.064283 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2742cf8-fdb1-4628-8e6f-6de0ea0e0b89 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.069731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 257/503] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:58.070502 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2742cf8-fdb1-4628-8e6f-6de0ea0e0b89 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.075168 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:58.115522 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82f7f-d284-4864-8948-75b88ace928c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.132484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:58.154655 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.155333 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.155617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.155887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.156724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:58.156959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.158541 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25952ce6-d094-4724-9c10-1fa9c8a3939d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.179168 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 247/504] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 06:00:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:58.180551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.180840 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.181115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.197220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2742cf8-fdb1-4628-8e6f-6de0ea0e0b89 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.204174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:58.206219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:58.239211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.239845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.240134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.240421 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.290521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 248/505] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:58.292276 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.329120 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.331248 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.358767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:58.385250 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:58.408159 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98de3eb-80d2-4ab8-9259-bcc27dfa41d1 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.445773 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.446333 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.446615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.446886 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.449675 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 258/506] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:00:58.457023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.457651 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.459133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:58.474959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:58.476067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:58.477075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:58.477075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:58.477075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:58.477075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 249/507] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:58.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.588872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6010f-059a-494f-ad0d-918506657cc7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.607489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:58.657511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9516-6724-4784-9519-0e099186bc6e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.659604 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.660227 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.660505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.660775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.661430 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:58.661735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.663129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.697310 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 250/508] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:00:58.701612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.702254 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.702543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.702823 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.703171 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.713695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.715156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.725537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:58.734490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:58.740557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:58.741544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:58.741544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:58.741544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:58.741544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 259/509] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:58.741544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.844000 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070a116-d397-443f-88c7-3b06493444d0 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.868497 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ec35a-6d33-444c-9fbd-fac89014d5cc None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:58.888499 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568cce-63c4-4367-b815-350613746895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.889144 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568cce-63c4-4367-b815-350613746895 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.889586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568cce-63c4-4367-b815-350613746895 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.889857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568cce-63c4-4367-b815-350613746895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.898357 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14568cce-63c4-4367-b815-350613746895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.900337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14568cce-63c4-4367-b815-350613746895 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.910374 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:58.911153 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:58.911587 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:58.911960 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-14568cce-63c4-4367-b815-350613746895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:58.914372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:58.915839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 251/510] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:58.917453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568cce-63c4-4367-b815-350613746895 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.928760 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:58.930948 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:00:58.953485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:58.955492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:00:58.957353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:00:58.957353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:00:58.957353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:58.957353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 260/511] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:58.957353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:58.965152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.013814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.049171 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568cce-63c4-4367-b815-350613746895 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.064537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6077-d5ca-40db-b11a-9b0473b2d540 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.082841 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.083485 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.083771 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.084042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.084680 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:59.084981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.086344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.098236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.098845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 252/512] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:59.099027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.099474 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.099643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.100019 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.100867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:59.101338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.103220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4f31b-ccc4-4f7a-98cf-c34b1a356d92 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.126425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 261/513] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:59.126425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b285c-9247-4b95-9633-550db2c261cd None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.126425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b285c-9247-4b95-9633-550db2c261cd None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.126425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b285c-9247-4b95-9633-550db2c261cd None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.126978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 262/514] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:59.128693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf9d1-2d1b-42b8-a2dc-1b7bad46dcc6 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.129157 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf9d1-2d1b-42b8-a2dc-1b7bad46dcc6 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.129610 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf9d1-2d1b-42b8-a2dc-1b7bad46dcc6 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.131956 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 263/515] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:00:59.133176 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf9d1-2d1b-42b8-a2dc-1b7bad46dcc6 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.211428 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.220850 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fb824-81ab-4b3e-99a4-c16e3a5e11f8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.246308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf9d1-2d1b-42b8-a2dc-1b7bad46dcc6 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.254826 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.255464 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.255741 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.256009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.256648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:59.256948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.258321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.259134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.270080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 253/516] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:59.271796 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.281393 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.282758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.283448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.284485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.291344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.292802 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-2a9378cf-0d1a-4cb7-bc8e-9f84a81aa791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:00:59.304848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:59.305825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:59.307039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 264/517] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:00:59.307606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eafde-7dfb-411e-950d-46a30e8eedba None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.307680 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eafde-7dfb-411e-950d-46a30e8eedba None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.308250 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eafde-7dfb-411e-950d-46a30e8eedba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.325469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-ca6eafde-7dfb-411e-950d-46a30e8eedba None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:59.327181 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-ca6eafde-7dfb-411e-950d-46a30e8eedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:59.371431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e6321-0213-4061-9290-6ea76fa7c8d2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.392551 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 265/518] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:59.393763 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57275b-9a53-456f-b403-637f71327daa None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.394046 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57275b-9a53-456f-b403-637f71327daa None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.394319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57275b-9a53-456f-b403-637f71327daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.414207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.414879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.415160 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.415473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/9dcc0e6649de439eb130a3b5f5bc1b91/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.417433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=9dcc0e6649de439eb130a3b5f5bc1b91, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.418860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.425786 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-9d57275b-9a53-456f-b403-637f71327daa None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:00:59.426983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-9d57275b-9a53-456f-b403-637f71327daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:00:59.477592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 254/519] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:00:59 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/9dcc0e6649de439eb130a3b5f5bc1b91/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:00:59.478594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.502000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 266/520] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:00:59.503190 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57275b-9a53-456f-b403-637f71327daa None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.509795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.581698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.689811 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec98c-7e8a-45a7-bb37-68a0f8f761a2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.708868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57275b-9a53-456f-b403-637f71327daa None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.729543 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.730082 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.730363 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.730635 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.731283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.733627 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.753519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.754279 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.754616 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.754949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.757841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:59.757841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.757841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.774947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 267/521] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:59.774947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.803369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 255/522] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:00:59 2022] POST /identity/v3/users => generated 254 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:00:59.804486 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:00:59.888615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.906346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:00:59.920279 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcff44-1b84-444d-be5f-90f6ab2c9998 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.933018 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27165f3-9711-4f1e-a0f4-be190a51e90a None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:00:59.970937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.972115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.972585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.973033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.973817 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.975580 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.983066 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:00:59.983768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:00:59.984124 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:00:59.984470 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:00:59.985286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=1e93da8c826f427a8a7fb357f6dcfc67) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:00:59.985670 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:00:59.987208 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:00:59.990016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:00:59.991121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:00:59.992115 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:00:59.999356 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 256/523] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:00:59 2022] GET /identity/v3/role_assignments?user.id=1e93da8c826f427a8a7fb357f6dcfc67&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:00:59.999356 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:00.000863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 268/524] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 06:00:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:00.008420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:00.140457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44557165-9189-4ce0-9c99-da4b104a9ffe None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:00.151116 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ac0ff-9c54-46c2-914b-7e3f42710d29 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:00.184033 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:00.191668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:00.192695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:00.198775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:00.198775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] PATH_INFO: `/v3/services/network` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:00.198775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:00.198775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:00.199422 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:00.200432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:00.200753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:00.201030 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:00.201977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:00.202280 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:00.203710 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:00.209480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 269/525] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 06:01:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:00.210146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:00.211268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:00.212218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 257/526] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:00:59 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:00.220027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:00.220577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:00.251533 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:00.350495 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dff7f-e7de-4cf6-bc78-29c44075867e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:00.355931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51f0b-3bf7-412e-a905-0ca9d55a66f9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:00.397929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:00.398612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:00.398902 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:00.399012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:00.399224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:00.399632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:00.399824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] RBAC: Adding query filter params (name=network) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:00.399923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:00.400129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:00.400246 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:00.400834 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:00.401182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:00.401511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:00.402816 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad20e-e07b-48c6-b6db-45f6af64b6cc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:00.407753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 258/527] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:00.413559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:00.414609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 270/528] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 06:01:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:00.496936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:00.543748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d010f-f580-4f31-8328-479e9b19465c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:00.589677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:00.590301 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:00.590579 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:00.590848 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:00.592051 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] RBAC: Adding query filter params (type=network) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:00.592355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:00.593798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1955c51-1330-4c89-ac7c-428525d48e71 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:00.605087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 259/529] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:01.926042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9927-030f-4851-875a-f8c0abd4eace None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:01.926607 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9927-030f-4851-875a-f8c0abd4eace None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:01.926607 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9927-030f-4851-875a-f8c0abd4eace None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:01.928227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 271/530] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:01.933550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c6e4-f4db-4d97-b331-c8ed6ec11094 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:01.933957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c6e4-f4db-4d97-b331-c8ed6ec11094 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:01.934129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c6e4-f4db-4d97-b331-c8ed6ec11094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:01.960297 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-0765c6e4-f4db-4d97-b331-c8ed6ec11094 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:01.961508 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-0765c6e4-f4db-4d97-b331-c8ed6ec11094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:02.046563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 260/531] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:01:02.377718 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9927-030f-4851-875a-f8c0abd4eace None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:02.505919 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:02.532541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9927-030f-4851-875a-f8c0abd4eace None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:02.575918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:02.576556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:02.576834 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:02.577106 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:02.595936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:02.600974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:02.602012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:02.602935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:02.603663 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb81a6-a954-4d75-b32f-a790d986ad24 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:02.603663 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb81a6-a954-4d75-b32f-a790d986ad24 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:02.603663 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb81a6-a954-4d75-b32f-a790d986ad24 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:02.603859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 272/532] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:02.604456 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 261/533] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:02.608530 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:02.608951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:02.609235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:02.609512 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:02.627996 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:02.629166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:02.703481 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:02.747940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 262/534] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:01:02.775498 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8b7d3-29ab-4631-bca8-cc3f45f3ba38 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:02.822465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:02.823070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:02.823383 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:02.823656 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:02.824320 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:02.824621 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:02.826020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5e78-b23c-44ad-8a18-b41f141c11d8 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:02.838974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 273/535] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:01:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:02.844861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:02.975023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448e54d-013b-4525-85c4-e15dcecae397 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.022374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.023074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.023381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.023651 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.036352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.038105 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:03.057483 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:03.058550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:03.059549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:03.059549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:03.059549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:03.062194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 263/536] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:03.066215 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbc777-d74e-40a8-92c0-fb9f14862e95 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.066681 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbc777-d74e-40a8-92c0-fb9f14862e95 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.066767 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbc777-d74e-40a8-92c0-fb9f14862e95 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.067838 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.068379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 274/537] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:03.073314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.073601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.073877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.102586 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.113033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:03.115087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:03.119658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.210700 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 275/538] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 06:01:03.211631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870d81e-82e7-419e-859e-1f0cabdf0646 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.211756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.252191 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.252767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.253049 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.253319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.253964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:03.254262 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.255752 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:03.269867 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 264/539] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:03.275522 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.328019 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2d9b-90f6-4d41-97d7-16d9ad5dfe5a None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.368590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18629704-be77-40c2-812b-88db459926ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.369339 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629704-be77-40c2-812b-88db459926ce None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.369682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629704-be77-40c2-812b-88db459926ce None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.370053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18629704-be77-40c2-812b-88db459926ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.373814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.383336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18629704-be77-40c2-812b-88db459926ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-18629704-be77-40c2-812b-88db459926ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:03.388537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:03.391074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:03.392764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 276/540] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:03.394267 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.394404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bf5e9-a44a-43e8-bf61-805719ddfac5 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.394404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bf5e9-a44a-43e8-bf61-805719ddfac5 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.394652 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bf5e9-a44a-43e8-bf61-805719ddfac5 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.398552 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 277/541] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:03.398552 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bf5e9-a44a-43e8-bf61-805719ddfac5 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.398885 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fcd10-70b6-4a3f-9a4e-0a3b5f940fda None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.440772 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.441466 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.441741 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.442010 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] PATH_INFO: `/v3/users/nova` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.451072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.452559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-39a709aa-7067-44ed-b9fe-cfcacbaab894 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:03.469316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:03.471048 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 265/542] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.472824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.473731 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.492336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:03.493457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:03.514744 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bf5e9-a44a-43e8-bf61-805719ddfac5 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.549901 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.550503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.550785 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.551053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.551748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:03.552044 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.553387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d013352-e0e3-48e0-8238-6ff034222329 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:03.561492 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 266/543] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:01:03.561983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 278/544] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:03.563100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b70c39-fba2-4ba0-aad8-dcfce6979228 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.563415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b70c39-fba2-4ba0-aad8-dcfce6979228 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.563691 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b70c39-fba2-4ba0-aad8-dcfce6979228 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.569344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 279/545] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:03.569344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b70c39-fba2-4ba0-aad8-dcfce6979228 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.569541 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.633494 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.656493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.675182 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e46202-abed-418f-84e7-d973875dcb0d None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.679141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b70c39-fba2-4ba0-aad8-dcfce6979228 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.712417 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.713439 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.713962 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.714256 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.714601 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.714877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.715156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.715457 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.716739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:03.717204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.718778 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f39018-5c82-496c-809f-c90583bb63d8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:03.726294 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:03.731546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:03.733366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:03.734304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 280/546] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:03.735227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.746711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 267/547] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:03.746711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.746711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.746711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.765350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:03.766460 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:03.837221 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bed37-12dd-4787-a61d-4cf23cadad29 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.838055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 268/548] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 06:01:03.838885 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.854477 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.874543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.875180 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.875621 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.875900 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.890744 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.892277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:03.910897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:03.912096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:03.918349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:03.918349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:03.918349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:03.918349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 281/549] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:03.918349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:03.924657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:03.955824 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0236f-21cd-4458-8a0f-ebf846ccaa57 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:03.995417 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:03.996170 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:03.996546 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:03.996822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:03.997511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:03.997814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:03.999169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.007821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 269/550] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:04.009035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.021708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43f639-d8de-4ebd-bd4f-1ca6c052eedb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.055897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.056743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.057152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.057316 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.098278 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:04.123083 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4906ee-4c80-4d71-89c0-55fdfbc1337d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.128154 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.129607 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.156692 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.157330 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.157604 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.157873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.158512 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.159893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.163194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:04.227387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 282/551] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 06:01:04.228236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.255487 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 270/552] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:04 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:04.256321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.301340 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:04.336714 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a360ca-e84d-4e95-b99d-a27370d152fb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.380941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.381572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.381854 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.382125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.382797 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:04.383101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.384546 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.387446 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deff22-fc5e-4083-ba2c-63e20c99cccb None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.402493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 283/553] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:04.403699 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.433995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.435759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.435759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.435965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.447059 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.449232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:04.468511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:04.469577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:04.471157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:04.471157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:04.471157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:04.471157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 271/554] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:04.471157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.518701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58879996-f9d7-4196-8d9f-d826c30cdcf9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.518916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:04.545398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:04.554951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.555527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.555807 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.556078 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.557151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.558393 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.581864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02356367-3578-45d2-8fa1-c2a4cbdb8adb None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.614968 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.615632 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.615912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.616183 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.627079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.628562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:04.642525 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:04.643780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:04.644800 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:04.644800 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:04.644800 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:04.644800 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 272/555] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:04.644800 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.666131 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 284/556] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:04 2022] POST /identity/v3/services => generated 240 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:04.666977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.804947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:04.813693 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6cdea-cb69-42ae-904d-671e834c2af3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.823612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b31f3-cadc-432f-814c-593280e709c7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:04.855961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.856612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.857427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.857700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.864025 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:04.865454 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:04.865741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:04.866011 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:04.866661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:04.866961 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.868253 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:04.868388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.869992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:04.882033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 285/557] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:04.882679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:04.883975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:04.885211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:04.885898 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.888559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 273/558] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:04.894078 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:04.932968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.003111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04aafed-b7d2-4ed2-b303-1a489692a757 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.016813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ea5d3-249d-443a-9cd1-ce80eab5bb2e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.044200 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.044825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.045110 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.045389 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.046105 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.046440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.047830 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.061017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.061954 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.062412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 286/559] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.062679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.063249 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.063478 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.071982 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.073683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:05.087117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:05.088216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:01:05.089242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:05.092437 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 274/560] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:05.093503 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.114593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.183231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25130b23-0e7f-432b-a03b-42a0cd0f9f38 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.185099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.219391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a56284-7b1a-4788-a0a0-e5731b17d322 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.222925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.223639 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.223994 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.224337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.225143 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.225519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.227022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.251346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 287/561] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.251346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.274527 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.275325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.275799 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.276140 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.285052 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.287195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:05.300004 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:05.300966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:05.301933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:05.302431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 275/562] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:05.302997 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.364761 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95a97-0bad-42f4-a77d-d0c56807ed5f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.392248 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.411159 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.411845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.412277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.412559 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.413300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.413609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.415042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.422282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f13ed-9002-4751-9abb-06cb19ba1a88 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.441518 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 288/563] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.442538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.463778 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.467060 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.468024 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.468498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.470811 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.482846 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.484579 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:05.501206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:05.502482 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:05.503557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:05.503557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:05.503557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:05.505325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 276/564] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:05.506537 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.555039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c0f2d-fc0d-4a13-88d6-2e994c06d421 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.592834 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.593452 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.593738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.594016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.594723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.595058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.596555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.609241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 289/565] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.610096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.635653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ae504-fdf0-41d0-87aa-0028f23ba777 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.670487 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.681184 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.682314 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.682747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.683521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.696210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.698085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.698924 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:05.715199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:05.716786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:05.717791 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:05.717791 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:05.717791 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:05.724574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 277/566] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:05.724574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.727133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29e020-5ad9-4a8a-a565-42c21bb36062 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.764344 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.764983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.765868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.765868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.766177 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.766543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.768027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.784564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 290/567] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.784564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.853437 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a32fe-2850-48b0-a8a8-3721613ad7ae None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.892547 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ca291-ba96-4938-a14e-c9058a694314 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:05.894599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.895514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.895941 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.896393 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.897273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=9dcc0e6649de439eb130a3b5f5bc1b91) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.897739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.899353 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.906851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 278/568] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/role_assignments?user.id=9dcc0e6649de439eb130a3b5f5bc1b91&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.907921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.934830 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:05.935454 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:05.935845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:05.936227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:05.937083 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:05.937521 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:05.939334 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:05.947090 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:05.953586 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 291/569] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:05.960182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:05.972570 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:06.084563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c533b74-34dc-41be-b56c-9e7feac24d46 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:06.141909 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325249dd-dd11-4663-9709-ecb196a30617 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:06.147559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:06.148147 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.148426 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.148697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.157685 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:06.159623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:06.174498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:06.175642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:06.176622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:06.179812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 279/570] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:06.186235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:06.187917 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:06.188620 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.188903 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.189171 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.196717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:06.198358 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:06.212255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:06.213289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:06.214314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:06.218325 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 292/571] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:06.219825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:06.307472 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:06.332079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15376b-bdd3-4e5b-89a5-223852bd31ed None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:06.335589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:06.376000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a86fb-147a-4eb5-a17b-5995b64c43e0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:06.393296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:06.393911 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.394192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.394455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.395150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:06.395508 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:06.396966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:06.421650 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 280/572] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:06.428807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:06.443417 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:06.444285 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.444589 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.444863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.445572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:06.445878 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:06.447336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:06.477016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 293/573] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:06.481336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:06.629035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b037d0-1e3a-4123-9a1c-4ddceaf864a8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:06.679346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388de16-6452-4f55-9ae2-f5332d00be83 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:06.680912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4772b58-4ab0-41c6-bc9e-5188f9995731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:06.681066 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4772b58-4ab0-41c6-bc9e-5188f9995731 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.681315 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4772b58-4ab0-41c6-bc9e-5188f9995731 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.681848 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4772b58-4ab0-41c6-bc9e-5188f9995731 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/6f52fc5710c944b1a0ef03df0b436271/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.697169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4772b58-4ab0-41c6-bc9e-5188f9995731 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=6f52fc5710c944b1a0ef03df0b436271, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:06.698627 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4772b58-4ab0-41c6-bc9e-5188f9995731 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:06.699541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:06.719337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702fcf0-e677-40c5-a364-31dd840bf966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:06.720444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fcf0-e677-40c5-a364-31dd840bf966 None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.720951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fcf0-e677-40c5-a364-31dd840bf966 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.721476 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fcf0-e677-40c5-a364-31dd840bf966 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/1e93da8c826f427a8a7fb357f6dcfc67/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.723712 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702fcf0-e677-40c5-a364-31dd840bf966 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=1e93da8c826f427a8a7fb357f6dcfc67, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:06.725340 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702fcf0-e677-40c5-a364-31dd840bf966 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:06.781126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 281/574] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:01:06 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/6f52fc5710c944b1a0ef03df0b436271/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:01:06.782697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0d7ff-fbd3-44a7-8a83-89521eef50ed None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.783134 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0d7ff-fbd3-44a7-8a83-89521eef50ed None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.783580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0d7ff-fbd3-44a7-8a83-89521eef50ed None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.785419 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 282/575] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:06.800809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec57fd-7cfb-4e2f-8447-0c280cc16f9d None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:06.801070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec57fd-7cfb-4e2f-8447-0c280cc16f9d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:06.801438 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec57fd-7cfb-4e2f-8447-0c280cc16f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:06.804968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 294/576] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:01:06 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/1e93da8c826f427a8a7fb357f6dcfc67/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:01:06.836062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-26ec57fd-7cfb-4e2f-8447-0c280cc16f9d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:06.837308 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-26ec57fd-7cfb-4e2f-8447-0c280cc16f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:06.854489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:06.962311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 283/577] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:07.070838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650dc89d-4b93-4cd5-9e4a-2f0877c21053 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.071198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650dc89d-4b93-4cd5-9e4a-2f0877c21053 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.071418 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650dc89d-4b93-4cd5-9e4a-2f0877c21053 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.073014 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 295/578] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:07.078918 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.079206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.079576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.100565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:07.101790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:07.173684 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 284/579] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:07.277189 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650dc89d-4b93-4cd5-9e4a-2f0877c21053 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:07.412233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650dc89d-4b93-4cd5-9e4a-2f0877c21053 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:07.484391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:07.487885 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:07.488532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.488809 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.489079 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.489671 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:07.491109 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:07.504129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:07.505319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:07.506328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 296/580] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 06:01:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:07.512438 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:07.562898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:07.583773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:07.605674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5adb61-606a-48d7-b7cd-126b8c492461 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:07.642354 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:07.643006 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.643288 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.643587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] PATH_INFO: `/v3/services/network` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.644183 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:07.645609 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:07.654899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:07.656035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:07.656944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 285/581] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:07 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:07.662001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:07.684612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2023a1-8f02-4cd8-990e-3debb9e856bf None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:07.752774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:07.753394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.753673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.753952 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.754594 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:07.754897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:07.756365 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:07.761404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 297/582] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 06:01:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:07.767896 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:07.843691 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107e42f-7e6d-4e75-a9d5-ca607d4cb168 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:07.909425 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:07.911397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.912065 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.913582 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.914487 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] RBAC: Adding query filter params (name=network) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:07.915152 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:07.917494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:07.923748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:07.931418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 286/583] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:07.933173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e258ba7-bd09-47da-8235-046eb9e2d06a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:07.937161 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:07.944173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:07.991230 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:07.991883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:07.992209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:07.992487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:07.993183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:07.993489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:07.999319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:08.000795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 298/584] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 06:01:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:08.012304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:08.075190 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bda30-885b-4cb2-a906-4993a79f6b22 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:08.111668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:08.113133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.113684 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.114168 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.115025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] RBAC: Adding query filter params (type=network) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:08.115580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:08.117967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:08.124763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 287/585] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:08.130194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:08.200848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2b47-d282-48c4-93b3-a0757498ecaa None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:08.253280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:08.269791 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:08.275102 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.275679 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.276206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.277133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] RBAC: Adding query filter params (interface=public, service_id=c544e5bc83e94983b1928ce9458ed6ee, region_id=RegionOne) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:08.277657 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:08.279815 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:08.296836 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 299/586] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:08 2022] GET /identity/v3/endpoints?service_id=c544e5bc83e94983b1928ce9458ed6ee&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:08.302979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:08.338769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca8264-5d50-4771-b910-5ab8eaa049ed None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:08.365042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:08.408020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:08.408662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.408948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.409225 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.409936 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] RBAC: Adding query filter params (interface=public, service_id=4695c95f24d3405c976a12ec1d37b24e, region_id=RegionOne) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:08.410250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:08.411657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:08.429301 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 288/587] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:08 2022] GET /identity/v3/endpoints?service_id=4695c95f24d3405c976a12ec1d37b24e&interface=public®ion_id=RegionOne => generated 194 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:08.435121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:08.446810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cacab-9da1-44cb-a155-823126597d88 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:08.498269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf6ae7-7619-41ed-82e3-15b2ca0e2f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:08.498883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf6ae7-7619-41ed-82e3-15b2ca0e2f60 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.499167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf6ae7-7619-41ed-82e3-15b2ca0e2f60 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.499476 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf6ae7-7619-41ed-82e3-15b2ca0e2f60 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.500088 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf6ae7-7619-41ed-82e3-15b2ca0e2f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:08.501634 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf6ae7-7619-41ed-82e3-15b2ca0e2f60 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:08.507264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 300/588] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:08 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:08.668674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c5c80-0e9e-4186-a567-95d446620f6f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:08.710357 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:08.720295 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2706a2-87df-4e04-bd07-40419e932f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:08.720295 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2706a2-87df-4e04-bd07-40419e932f55 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.724983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2706a2-87df-4e04-bd07-40419e932f55 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.724983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2706a2-87df-4e04-bd07-40419e932f55 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.724983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2706a2-87df-4e04-bd07-40419e932f55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:08.724983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2706a2-87df-4e04-bd07-40419e932f55 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:08.735723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 289/589] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:08 2022] GET /identity/v3/services => generated 772 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:08.859541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515020b-02d1-4dd8-b50b-341caf3e5af8 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.860194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515020b-02d1-4dd8-b50b-341caf3e5af8 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.860672 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515020b-02d1-4dd8-b50b-341caf3e5af8 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.862234 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 301/590] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:08.868398 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e2fd0-9974-4783-a66a-7758ddb54c10 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:08.868750 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e2fd0-9974-4783-a66a-7758ddb54c10 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:08.869032 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e2fd0-9974-4783-a66a-7758ddb54c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:08.892348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-634e2fd0-9974-4783-a66a-7758ddb54c10 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:08.893573 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-634e2fd0-9974-4783-a66a-7758ddb54c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:08.986085 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 290/591] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:09.302209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515020b-02d1-4dd8-b50b-341caf3e5af8 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:09.378279 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242a17f-8870-4272-b316-f50f08cb846d None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:09.378823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242a17f-8870-4272-b316-f50f08cb846d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:09.378823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242a17f-8870-4272-b316-f50f08cb846d None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:09.380557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 291/592] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:09.386724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bf09-e487-44c8-955c-53d12ca7bf5e None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:09.387077 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bf09-e487-44c8-955c-53d12ca7bf5e None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:09.387457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bf09-e487-44c8-955c-53d12ca7bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:09.420222 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-2a12bf09-e487-44c8-955c-53d12ca7bf5e None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:09.428876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-2a12bf09-e487-44c8-955c-53d12ca7bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:09.455064 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:09.469060 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515020b-02d1-4dd8-b50b-341caf3e5af8 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:09.517282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:09.517943 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:09.518196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:09.518463 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:09.519050 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:09.520468 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 292/593] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:09.520601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:09.527517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc20985-d6f9-4b9f-928b-6a9556eba8a1 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:09.527517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc20985-d6f9-4b9f-928b-6a9556eba8a1 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:09.527517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc20985-d6f9-4b9f-928b-6a9556eba8a1 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:09.530869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 293/594] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:09.540112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:09.541162 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:09.542167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:09.543530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:09.543530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:09.543720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:09.547016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 302/595] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 06:01:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:09.547875 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:09.577624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:09.578740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:09.670516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:09.703997 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 294/596] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:09.760592 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81b980-e729-4322-bc5a-9eb4918e7b59 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:09.805253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:09.805864 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:09.806145 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:09.806421 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:09.807335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:09.807644 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:09.809014 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:09.821428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 303/597] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:09.826971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:09.835795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:09.893202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:09.961734 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060162f9-af39-4a4c-b5f0-a49027787ee4 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:09.999035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.000069 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.000570 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.001043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.001887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:10.002443 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:10.004039 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:10.009290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 295/598] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:10.025765 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.057949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb9c62-97b1-49b5-a2de-0c2135e6ab6c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.102883 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:10.108586 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.109247 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.109534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.109806 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.131111 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:10.144747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:10.145906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:10.146861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 304/599] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:10.154942 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.173025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e2fda-95c7-46d8-a5ad-fffab3a8d8eb None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.210970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.211979 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.212428 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.212947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.225876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:10.231391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:10.232761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:01:10.233781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:10.236244 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 296/600] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:10.244463 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.289228 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:10.304619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3c754-1e2f-4ecb-ab61-9e60f0280c43 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.368758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.369288 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.369568 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.369838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.370508 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:10.370811 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:10.375343 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:10.384071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 305/601] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:10.390414 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.407672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9735ffa-bc3d-4ed6-81b0-b9cd44e510ad None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.427289 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:10.494689 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.496328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.497002 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.499650 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.500558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:10.501763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:10.503458 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:10.518767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 297/602] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:10.524796 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.576687 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf965b-dca7-40f3-bb0c-c0f84fe37eb7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.644074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.645159 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.646184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.646822 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.671223 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:10.673035 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:10.689429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:10.702068 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:10.712735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:10.713872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:10.713872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:10.713872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:10.713872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 306/603] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:10.713872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.728267 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73967bb3-0fed-4e9e-80c0-c3c9cd525bfb None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.791616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.792262 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.792542 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.792806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.815255 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:10.816733 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:10.841185 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:10.842300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:10.843269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:10.843269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:10.843269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:10.848887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 298/604] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:10.855007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:10.894532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94a8ef-408f-448f-a0c6-97ecc218c718 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:10.897688 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:10.946396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:10.947008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:10.947289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:10.947575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:10.948222 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:10.948534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:10.949891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:10.968918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 307/605] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:10.974779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.030591 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0ac12-487c-4c0f-932b-b25f4e1fb822 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.079381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.080062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.080346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.080618 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.081278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:11.081579 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:11.083073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:11.096039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:11.099513 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 299/606] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:11.106300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.183364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b196c0-137b-4a71-a169-793aad841d3b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.242890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.246218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.246503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.246788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] PATH_INFO: `/v3/users/swift` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.260030 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:11.260214 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:11.262155 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-cfe5b1f9-5984-4a39-8cc2-72a5354ab198 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:11.278605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:11.284916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 06:01:11.285976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:11.286596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c473fc-b877-40ec-ba9a-e5b9f15a839d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.286684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 308/607] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:11.286684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388d984-d8d4-4568-a514-74e30ea13c20 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.286684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388d984-d8d4-4568-a514-74e30ea13c20 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.286684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388d984-d8d4-4568-a514-74e30ea13c20 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.288576 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 309/608] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:11.289397 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388d984-d8d4-4568-a514-74e30ea13c20 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.324910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.326204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.326547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.326929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.335816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:11.337479 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-c5aa7257-7014-4ec7-bbdd-c4b6befbc7e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:11.351985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:11.354088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:11.357025 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 300/609] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:01:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:11.364873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.364873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.364873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.382148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:11.383545 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:11.413626 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388d984-d8d4-4568-a514-74e30ea13c20 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.441847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:11.459409 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 301/610] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:11.461388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.472056 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.472899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.473345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.473761 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.474746 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:11.475052 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:11.479997 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a12361-9b60-43f9-834c-853abfdd2a15 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:11.501211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 310/611] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Oct 17 06:01:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:11.502946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ad6a0-7009-4c71-9405-60ce2e9ae71d None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.503326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ad6a0-7009-4c71-9405-60ce2e9ae71d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.503661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ad6a0-7009-4c71-9405-60ce2e9ae71d None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.509562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 311/612] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:11.509562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ad6a0-7009-4c71-9405-60ce2e9ae71d None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.576167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:11.583865 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7b556-ab7e-4f68-afd1-31e6d0d8dad9 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.625642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.626707 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.627536 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.628295 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.629370 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:11.629917 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:11.632133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71185209-e0fc-4ffa-854f-a095aa839857 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:11.634129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ad6a0-7009-4c71-9405-60ce2e9ae71d None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.654090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 302/613] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 06:01:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:11.655784 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.656626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.657247 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.679008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.679673 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:11.679807 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.680055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.680406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.682118 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:11.735196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:11.757678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:11.759070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 303/614] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:11.759263 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:11.760280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.823971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:11.872637 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc175a-9d53-4cb4-a506-bb692ce23ffb None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:11.896217 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 312/615] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:01:11.897172 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:11.919242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:11.919976 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:11.920259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:11.920537 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:11.986151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.003445 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.005949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.009039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee42ca-3690-4e0c-b45e-0b36932f2576 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.045857 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.046495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.046774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.047053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] PATH_INFO: `/v3/services/network` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.047683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.049049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:12.058719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:12.060003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:12.060924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 313/616] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:11 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:12.061795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.073452 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.117897 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 304/617] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:01:12.123243 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.166577 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d809d-8908-4189-92cb-3c292f8a6dad None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.202273 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.202909 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.203191 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.203507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.214987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.216611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.218494 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:12.232390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:12.236459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:12.237564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:12.237564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:12.237564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:12.237564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 314/618] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:12.237564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.240981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962e5eb-ecf1-46a1-8935-8b50ce5b9156 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.281506 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.282748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.283413 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.283915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.284702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.286119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:12.297489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:12.298754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:12.299697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 305/619] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:12.301204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.328969 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.357235 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01183415-1ee4-4233-bc50-77fa6b727d96 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.394314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.395025 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.395404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.395752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.396465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] RBAC: Adding query filter params (name=network) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:12.396838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.398268 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.402832 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 315/620] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:12.404007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.410937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10015833-4299-4043-9ed2-a6ffdbf9400e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.444999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.445628 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.445907 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.446178 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.456499 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.457915 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.466329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:12.476879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:12.482434 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:12.483576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:12.483576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:12.483576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:12.483576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 306/621] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:12.483576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.505852 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1997e64-8c52-47ab-af37-e193b1068361 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.538929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.539570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.539853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.540128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.540773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:12.541081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.542454 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa1e8-8a64-4bed-800a-65e8264a8689 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.554987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 316/622] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:12.555284 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.556182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaddb23-8014-4adf-816a-1d128e6e6221 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.556464 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaddb23-8014-4adf-816a-1d128e6e6221 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.557002 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaddb23-8014-4adf-816a-1d128e6e6221 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.558349 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 317/623] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:12.558894 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaddb23-8014-4adf-816a-1d128e6e6221 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.585780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b259d3-eae3-43dc-b941-7bc90e0ebc83 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.621790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.622429 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.622718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.622992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.623689 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:12.623994 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.625356 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.629925 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 307/624] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:12.630678 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.661654 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaddb23-8014-4adf-816a-1d128e6e6221 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.678311 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.696062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.696694 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.696971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.697235 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.697988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] RBAC: Adding query filter params (type=network) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:12.698293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.699671 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.707595 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 318/625] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:12.708426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.730108 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d935-745f-4416-8b4f-27f323102ea8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.764156 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.764704 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.764992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.765264 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.765910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:12.766213 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.767626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2189a3-46f2-49ca-9bcd-385aeccdaf1e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.774389 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.779900 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 308/626] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:12.781141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.781498 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.781735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.800822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:12.802909 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:12.809737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae68455-481d-4999-a0c9-f30d7422f7b3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.843386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:12.844013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:12.844294 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:12.844565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:12.851201 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:12.852670 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:12.864326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:12.865455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:12.866450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 319/627] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:12.867007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.872608 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 309/628] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:12.873431 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:12.919519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:12.978816 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96706c-49bd-4704-8578-9088284fa9ae None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:12.980338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e750bd0-e84c-443e-a033-44627a3bc9df None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.006282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:13.015568 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.015726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.015844 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.015937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.016017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.016106 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.016106 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.016250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.016691 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:13.016875 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.017066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.018183 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.018384 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.028152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 320/629] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Oct 17 06:01:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:13.028152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.115326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 310/630] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:13.115326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.132638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046ec4-1470-4a7c-ab6d-d181e6f55061 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.141650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:13.178044 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.178709 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.178989 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.179261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.188222 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.191220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:13.205051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:13.206133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:13.207156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 321/631] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:13.207766 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.245022 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cb898-f2da-435c-894a-53569ab84855 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.286420 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.286967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.287251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.287558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.288258 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:13.288556 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.289999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.302447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 311/632] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:13.303261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.319378 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2e480-fe75-4877-8f1d-d85dbac4cf6e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.357593 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.358117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.358398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.358668 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.359379 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.359781 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:13.360766 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.416226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262139f-7407-4c93-b797-4ec8ab7fda77 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.448953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.449558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.449840 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.450128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.451348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.452730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.473484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 322/633] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:13.473484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.493740 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:13.529005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 312/634] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:13 2022] POST /identity/v3/services => generated 269 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:13.536374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.601243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001c84c-a92a-4720-b929-4f35c4967d92 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.652313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.653284 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.653734 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.654247 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.655208 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:13.655793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.657475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.679685 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 323/635] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:13.681455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:13.695766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d310e34-bce0-4468-836e-2750ccd3947f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.699646 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:13.741075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.741672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.741952 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.742226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.742964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=1e93da8c826f427a8a7fb357f6dcfc67) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:13.743273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.744815 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65585096-1b3f-4485-919a-176432d7dc4f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.751087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 313/636] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:13 2022] GET /identity/v3/role_assignments?user.id=1e93da8c826f427a8a7fb357f6dcfc67&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:13.771112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:13.813852 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb0bd0-0f4d-4ad5-ba32-ede037433673 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:13.852881 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:13.853878 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:13.854448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:13.854932 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:13.855976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=6f52fc5710c944b1a0ef03df0b436271) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:13.857020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:13.858707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceea85-e20c-4acb-a5a9-07483bd934d6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:13.871482 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 324/637] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:13 2022] GET /identity/v3/role_assignments?user.id=6f52fc5710c944b1a0ef03df0b436271&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:15.709560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e8b85-da9d-48ca-ad81-3a2b906eb9c7 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:15.710088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e8b85-da9d-48ca-ad81-3a2b906eb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:15.710088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e8b85-da9d-48ca-ad81-3a2b906eb9c7 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:15.711741 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 314/638] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:15.718362 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88f9b-ebef-4a28-993e-28feb774f7ff None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:15.718878 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88f9b-ebef-4a28-993e-28feb774f7ff None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:15.718878 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c88f9b-ebef-4a28-993e-28feb774f7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:15.737863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-f3c88f9b-ebef-4a28-993e-28feb774f7ff None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:15.739064 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-f3c88f9b-ebef-4a28-993e-28feb774f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:15.807013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 325/639] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:15.828338 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d6a62-869d-4c8f-bf8a-bd0310580032 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:15.828659 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d6a62-869d-4c8f-bf8a-bd0310580032 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:15.828894 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d6a62-869d-4c8f-bf8a-bd0310580032 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:15.830383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 315/640] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:15.835692 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf28340-d01f-4f5d-8b14-3aa4ae86e04f None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:15.835979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf28340-d01f-4f5d-8b14-3aa4ae86e04f None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:15.836260 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf28340-d01f-4f5d-8b14-3aa4ae86e04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:15.855167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-bdf28340-d01f-4f5d-8b14-3aa4ae86e04f None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:15.856407 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-bdf28340-d01f-4f5d-8b14-3aa4ae86e04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:15.922381 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:15.931230 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 326/641] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:16.001909 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db1cd-7385-4e94-8ed7-2532c03aabcd None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.002097 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db1cd-7385-4e94-8ed7-2532c03aabcd None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.002361 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db1cd-7385-4e94-8ed7-2532c03aabcd None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.004017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 316/642] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:16.009503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b8c47-9586-4d75-a444-2998f5079cab None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.009795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b8c47-9586-4d75-a444-2998f5079cab None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.010073 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b8c47-9586-4d75-a444-2998f5079cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.029062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-539b8c47-9586-4d75-a444-2998f5079cab None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:16.030251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-539b8c47-9586-4d75-a444-2998f5079cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:16.098517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffd1e0-3e61-44de-ba11-4f1b79c70147 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.098842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffd1e0-3e61-44de-ba11-4f1b79c70147 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.099079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffd1e0-3e61-44de-ba11-4f1b79c70147 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.100722 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 317/643] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:16.103657 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 327/644] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:16 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:16.104101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffd1e0-3e61-44de-ba11-4f1b79c70147 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.107515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.108030 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.108658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.128363 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:16.129903 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:16.209898 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 328/645] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:16 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:16.211134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.222615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffd1e0-3e61-44de-ba11-4f1b79c70147 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.259736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.260293 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.260576 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.260848 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.261433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.262928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.272817 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:16.273958 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:16.275046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:16.276530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 318/646] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:16.283979 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.321122 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:16.332737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1337a0-ec71-4986-a971-3b87982f2721 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.371574 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.373029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.373513 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.373979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.374772 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.376398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:16.385717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:16.386743 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:16.387844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:16.390335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 329/647] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:16.391366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.401324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbfec4-1cb5-4b34-a0e3-6c6adbaa8d1f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.439965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.440585 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.440851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.441125 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.441754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:16.442056 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.443417 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.448438 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 319/648] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:16.454210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.503077 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107d9af-e0b9-4545-b48c-e54454007861 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.520444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:16.536950 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.537576 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.538643 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.538643 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.538946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.540341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:16.549500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:16.550759 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:16.551821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:16.553739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:16.553893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 330/649] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:16.553893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.557391 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d60cf-c177-49d2-a3ee-5a919d2f39a6 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.592591 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.593320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.593745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.594017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.594644 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:16.594970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.596472 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.600988 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 320/650] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:16.601809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.653451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78db88d-0474-46e5-9bf9-a239275d159b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.687603 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.688213 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.688494 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.688770 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.689404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:16.689708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.691072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.695638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 331/651] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:16.696248 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.716524 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af98e6d-1ced-46ad-81e6-b93671ec36b3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.748151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:16.757325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.758094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.758382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.758688 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.772099 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:16.775697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:16.776937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:16.778101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 321/652] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:16.778910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.804050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:16.816245 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0405d06-e53e-4747-89ce-9e0364aea2b9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.852797 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.853439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.853713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.853990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.854642 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:16.854954 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.856940 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.866665 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 332/653] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:16.866665 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.900278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66fbe0-0604-4823-901f-d85041e307a1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.937538 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:16.938299 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:16.938906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:16.939189 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:16.939994 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:16.940296 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:16.941642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:16.951369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 322/654] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:16.951369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:16.984063 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc23d9-b353-413c-ae83-63f8a8549ef0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:16.994899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:17.019703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.020313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.020594 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.020869 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.021541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:17.021841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.023251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.032729 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 333/655] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:17.033564 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:17.041232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:17.057814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67688038-1318-4cd0-9b6c-56837e643932 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.095366 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.095977 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.096253 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.096520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.097148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:17.097445 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.098786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.111349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 323/656] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:17.111349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:17.165293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c46a5-be9a-43a7-9a80-89d112b92cf3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.208427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.209302 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.209714 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.210066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.210922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] RBAC: Adding query filter params (interface=public, service_id=3ad26fa004d24c4c8e8925dfc572fee9, region_id=RegionOne) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:17.211329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.212961 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.223756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 334/657] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/endpoints?service_id=3ad26fa004d24c4c8e8925dfc572fee9&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:17.226062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:17.252679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16dd7c-cee9-4a8c-b58a-ddc4bd3f90d4 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.269246 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:17.294612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.295648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.296146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.296645 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.316062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.318144 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.328276 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:17.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:17.340705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:17.341702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:17.341702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:17.341702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:17.341702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 324/658] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:17.351346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:17.366785 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af9c5b-c4df-4da0-bc7f-1a5588d20eac None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.413356 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.413977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.414261 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.414539 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.415158 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.417460 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.425119 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 335/659] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/services => generated 1029 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:01:17.485965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dce7b-aa40-4c0d-a3e8-56491ff707bc None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.525337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.526404 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.526877 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.527349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.528870 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:17.529382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.530977 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.545141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 325/660] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:17.552441 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:17.666727 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:17.669774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da47db-5fcb-4472-bdb8-223f6f2b927f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.707351 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.708052 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.708336 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.708611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.716219 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.717678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:17.729565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:17.730598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:17.731810 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:17.735406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 336/661] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:17.737028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:17.802206 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:17.845971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0b642-9368-4be2-b7a0-0267d153f934 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:17.883345 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:17.883979 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:17.884315 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:17.884599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:17.885341 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:17.885652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:17.887098 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:17.910678 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 326/662] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:17.912553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:18.018346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dac366-ae27-49fd-ad22-0b83dc56e0dd None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:18.058288 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:18.059367 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.060394 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.060951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.072972 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:18.074720 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:18.089789 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:18.090876 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:18.091949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:18.091949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:18.091949 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:18.098056 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 337/663] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:18.100071 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:18.207097 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eb43b-78b7-465b-a89c-04b4c9131a82 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:18.225876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:18.244298 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:18.245262 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.245759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.246320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.247790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:18.248329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:18.249991 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:18.266814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 327/664] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:18.268497 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:18.285469 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:18.376881 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d49f-3d57-45ca-8cbd-49c74afd6ea0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:18.413407 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:18.414439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.415007 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.415542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.423536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:18.425284 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:18.437695 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:18.438747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:18.439756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:18.445625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 338/665] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:18.448650 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:18.558783 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1597d8-ee27-43f7-abda-901464f99aab None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:18.594139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:18.594755 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.595094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.595401 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.596149 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:18.596461 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:18.597901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6b43-2d9a-4cd7-9609-a6c58a9ec2b2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:18.615519 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 328/666] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:18.616848 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:18.689684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:18.721796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6561b-e842-45f3-a5d0-2f45b7950fe7 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:18.757592 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c545d-bf02-4e26-88fd-7df7402c9ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:18.758269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c545d-bf02-4e26-88fd-7df7402c9ad6 None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.758600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c545d-bf02-4e26-88fd-7df7402c9ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.758882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c545d-bf02-4e26-88fd-7df7402c9ad6 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/6f52fc5710c944b1a0ef03df0b436271/roles/8f11efc95ebc40fe9eb7752b60fab42f` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.761138 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c545d-bf02-4e26-88fd-7df7402c9ad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=6f52fc5710c944b1a0ef03df0b436271, role_id=8f11efc95ebc40fe9eb7752b60fab42f)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:18.763101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c545d-bf02-4e26-88fd-7df7402c9ad6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:18.843448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 339/667] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:01:18 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/6f52fc5710c944b1a0ef03df0b436271/roles/8f11efc95ebc40fe9eb7752b60fab42f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:01:18.920088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970b080-6ef8-4e43-afea-6dbab8d41072 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.920650 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970b080-6ef8-4e43-afea-6dbab8d41072 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.920650 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970b080-6ef8-4e43-afea-6dbab8d41072 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.922014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 329/668] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:18.927838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd528bf-5a47-4eb8-a59e-412a504e565d None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:18.928356 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd528bf-5a47-4eb8-a59e-412a504e565d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:18.928474 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd528bf-5a47-4eb8-a59e-412a504e565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:18.948851 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-0fd528bf-5a47-4eb8-a59e-412a504e565d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:18.950516 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-0fd528bf-5a47-4eb8-a59e-412a504e565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:19.030543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 340/669] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:19.303489 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970b080-6ef8-4e43-afea-6dbab8d41072 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:19.347716 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:19.375157 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff873f78-cb8b-4605-baea-d46778f8ada3 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:19.375627 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff873f78-cb8b-4605-baea-d46778f8ada3 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:19.376058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff873f78-cb8b-4605-baea-d46778f8ada3 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:19.377775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 341/670] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:19.383127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:19.383440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:19.383730 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:19.401780 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:19.402976 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:19.417893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970b080-6ef8-4e43-afea-6dbab8d41072 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:19.457858 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c671d3-5fb8-4d74-8df9-2b35f14be2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:19.458531 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c671d3-5fb8-4d74-8df9-2b35f14be2e6 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:19.458881 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c671d3-5fb8-4d74-8df9-2b35f14be2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:19.459254 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c671d3-5fb8-4d74-8df9-2b35f14be2e6 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:19.462769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c671d3-5fb8-4d74-8df9-2b35f14be2e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:19.462769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c671d3-5fb8-4d74-8df9-2b35f14be2e6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:19.479596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:19.490719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 342/671] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:19 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:19.557843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 330/672] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:19 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:19.760296 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:19.830400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d75e0-d675-4888-afc5-c54d4f43483c None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:19.830599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d75e0-d675-4888-afc5-c54d4f43483c None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:19.830894 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d75e0-d675-4888-afc5-c54d4f43483c None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:19.832515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 331/673] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:19.837658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:19.837946 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:19.838224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:19.857961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:19.859519 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:19.869639 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8a1d-23f2-4adb-a072-a0372d481fe1 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:19.913697 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:19.914735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:19.915202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:19.915697 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:19.916665 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:19.918224 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:19.942180 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 332/674] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:20.021549 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 343/675] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:19 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:20.217825 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:20.246658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:20.322263 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2e846-0f48-4dd7-a3c7-28d6d83087e0 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:20.363104 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:20.363779 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:20.364201 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:20.364481 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:20.365106 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:20.366470 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:20.376242 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:20.377304 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:20.378325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 333/676] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Oct 17 06:01:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:20.386116 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:20.439632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:20.505563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a61cfe-403e-433c-b87e-880516ae1364 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:20.546543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:20.547576 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:20.548094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:20.548593 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:20.549446 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:20.549982 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:20.551611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:20.557453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 344/677] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:20.563477 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:20.654548 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:20.666268 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07793347-28b7-43b9-80eb-6c7c355df280 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:20.700748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:20.701372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:20.701653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:20.701926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:20.702561 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:20.702864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:20.704328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4f68a-4e60-47d3-8e6d-ef1f43a7afd5 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:20.709476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 334/678] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:20.715337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:20.825216 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf10e4-e191-47ff-8c96-6bcb3faaa546 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:20.838728 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:20.863128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50391d-b217-4d38-b82f-cbe29449af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:20.863887 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50391d-b217-4d38-b82f-cbe29449af8c None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:20.864187 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50391d-b217-4d38-b82f-cbe29449af8c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:20.864535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50391d-b217-4d38-b82f-cbe29449af8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:20.865108 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50391d-b217-4d38-b82f-cbe29449af8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:20.866877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50391d-b217-4d38-b82f-cbe29449af8c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:20.946841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 345/679] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:21.109600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e934-f7a5-4337-ac20-05b1621ad193 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:21.110594 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e934-f7a5-4337-ac20-05b1621ad193 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:21.111108 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6e934-f7a5-4337-ac20-05b1621ad193 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:21.113042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 335/680] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:21.118954 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:21.119945 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:21.120441 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:21.141308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:21.142829 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:21.219258 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 346/681] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:21.491030 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6e934-f7a5-4337-ac20-05b1621ad193 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:21.597685 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6e934-f7a5-4337-ac20-05b1621ad193 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:21.632515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b94f3-0cb5-43e5-8fea-a4f889326c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:21.633530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b94f3-0cb5-43e5-8fea-a4f889326c86 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:21.634029 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b94f3-0cb5-43e5-8fea-a4f889326c86 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:21.634519 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b94f3-0cb5-43e5-8fea-a4f889326c86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:21.646792 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b94f3-0cb5-43e5-8fea-a4f889326c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-6e1b94f3-0cb5-43e5-8fea-a4f889326c86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:21.652860 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:21.654611 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:21.655874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:21.658653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 336/682] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:01:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:21.664829 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:21.767374 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:21.775054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12654ba-3a71-41bd-9905-2be8447a84f1 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:21.812833 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:21.814018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:21.814509 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:21.815053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:21.816000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:21.816509 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:21.819184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:21.828492 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 347/683] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 06:01:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:21.831352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c30ac-7033-46af-8e97-e341ebdaabc1 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:21.831352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c30ac-7033-46af-8e97-e341ebdaabc1 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:21.831352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c30ac-7033-46af-8e97-e341ebdaabc1 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:21.832658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 337/684] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:21.834341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:21.837092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:21.837379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:21.837953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:21.859669 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:21.861035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:21.871647 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:21.942450 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 338/685] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:21.964511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb91a69-0d9e-41f9-99f4-cd10233769db None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.003402 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.004424 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.004929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.005425 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.017652 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.019429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:22.034719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:22.035821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:22.036983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:22.036983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:22.036983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:22.038641 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 348/686] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:22.044501 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.150019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c362cb-799e-4154-82ac-c42cce4cdb07 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.185474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.186092 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.186375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.186651 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.187312 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:22.187624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.188983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.201038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 339/687] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:22.206566 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.219468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:22.236724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.240677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:22.327882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07a01-39e1-4835-9964-ed3f68ef5b47 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.364950 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513ea87-5538-4747-86bd-ceea53d66a49 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.367175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.368523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.368523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.368523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.376862 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.378523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-e3f21646-ebf3-4561-bbf0-13a592c469cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:22.390794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:22.396683 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 349/688] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df3177-0f43-44ca-8181-326aa0c4115a None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.397747 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df3177-0f43-44ca-8181-326aa0c4115a None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.398512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df3177-0f43-44ca-8181-326aa0c4115a None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.398512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 350/689] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:22.398512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df3177-0f43-44ca-8181-326aa0c4115a None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.411075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.412075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.412364 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.412802 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.413382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.414736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-bf7cb503-360c-4071-93fc-d40741277be6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:22.424037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:22.427657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:22.428728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 340/690] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:22.429596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12782847-a471-4383-86ac-2db356b10100 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.429596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12782847-a471-4383-86ac-2db356b10100 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.429596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12782847-a471-4383-86ac-2db356b10100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.447552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-12782847-a471-4383-86ac-2db356b10100 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:22.448735 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-12782847-a471-4383-86ac-2db356b10100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:22.479014 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:22.514838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df3177-0f43-44ca-8181-326aa0c4115a None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.517528 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:22.525514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 341/691] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:22.530692 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12782847-a471-4383-86ac-2db356b10100 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.563807 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.564548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.564922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.565287 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.566133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:22.566528 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.568070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.591929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 351/692] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:22.592952 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.663801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12782847-a471-4383-86ac-2db356b10100 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.699575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1b043-973e-4b4b-870a-a442558894e1 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.713369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.714119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.714659 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.715656 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.716751 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:22.717690 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.719549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.730829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 342/693] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:22.733058 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.736102 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:22.745249 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.745977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.748978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.748978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.813561 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:22.819825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.822108 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.857139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168beb93-09ee-4a16-bf24-3d16229f8055 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:22.894497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:22.895128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:22.895428 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:22.895698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:22.896321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:22.896622 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:22.897980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:22.903420 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 343/694] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:22.904212 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.934787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 352/695] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:01:22.935958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:22.964161 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.022987 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651db59-0f4f-428a-b333-c6bb3cd320ce None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.041035 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a641c-0a9f-4060-8a20-4fe1a6a4699f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.050383 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.062179 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.063008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.063284 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.063668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.064246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.065591 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:23.075261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:23.076374 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:23.082617 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 344/696] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:23.083522 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.083598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.083598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.083598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.083598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.083886 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] RBAC: Adding query filter params (interface=public, service_id=d4e7ea2c195f4a97a1a28e5692d519bf, region_id=RegionOne) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:23.083886 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.083886 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.089675 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 353/697] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:22 2022] GET /identity/v3/endpoints?service_id=d4e7ea2c195f4a97a1a28e5692d519bf&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:23.095468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.190664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d84fc6-f1ca-402a-9158-006d03e81203 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.195128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.205011 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0e86c-ebc9-4328-a717-1dddac4b20e3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.227225 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.227972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.228320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.228664 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.240220 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.240547 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.241164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.241445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.241648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.241756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.242341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:23.242651 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.244037 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.248640 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 354/698] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:23.249413 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-52169944-a963-40d8-9c8d-422a70d3fe2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:23.255999 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:23.257194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:23.258371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:23.258371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:23.258371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:23.258371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 345/699] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:23.258371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734a786-0442-46cb-8b09-eb59aa1f4672 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.258777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734a786-0442-46cb-8b09-eb59aa1f4672 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.258777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734a786-0442-46cb-8b09-eb59aa1f4672 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.260094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 346/700] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:23.260636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734a786-0442-46cb-8b09-eb59aa1f4672 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.304722 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.359454 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d2564-6ed5-475d-ad7d-37d55b3871f3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.363663 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734a786-0442-46cb-8b09-eb59aa1f4672 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.392924 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.393534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.393820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.394097 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.394708 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.396103 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.396944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.397542 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.397819 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.398089 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.398715 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:23.399017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.400385 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b557b-0408-44a5-8387-fbc39df5c2fc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.402135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 355/701] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:01:23.403598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.406317 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 347/702] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:23.407521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.407809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.408086 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.426712 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.428770 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:23.430020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:23.522171 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cdc2a-b6e1-4595-b072-60248a9da79e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.524275 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 348/703] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:23.525061 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.568063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.581165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.581409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.581774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.582963 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.582963 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:23.584681 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.584827 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.599674 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 356/704] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:23.605119 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.655164 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34d85e-cc73-4a75-a84f-07f7fb659b2b None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.697843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.698563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.699756 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.700744 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.701172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.702087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] RBAC: Adding query filter params (interface=public, service_id=499fcfce81c94cc3a53c9b269b3769b1, region_id=RegionOne) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:23.702562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.704149 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.710689 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 349/705] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/endpoints?service_id=499fcfce81c94cc3a53c9b269b3769b1&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:23.719719 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.728639 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a10c9-355e-4054-a357-d4c67e654858 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.773758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.774402 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.774684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.774959 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.786282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.788808 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:23.803731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:23.804936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:23.806022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 357/706] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:23.806573 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.832113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:23.836827 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2eb75-ca08-4a72-843c-321c371e277c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.875068 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.875736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.876023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.876300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.876902 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.878263 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:23.895330 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 350/707] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 06:01:23.895330 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:23.936535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917d60a-745f-43af-b412-6d8081bfee51 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:23.984824 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:23.985412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:23.986340 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:23.986615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:23.987393 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:23.989280 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:24.002095 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:24.003257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:24.004209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 358/708] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:24.011218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:24.015632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:24.036219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786cfbd-db0f-4976-aed5-b9fee73ba3ed None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:24.084158 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:24.084158 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:24.084723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:24.085566 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:24.086675 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:24.087087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:24.088707 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:24.109890 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 351/709] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:24.111836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:24.135506 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84712400-5417-43be-8ee2-5082d71112cf None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:24.141072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:24.184265 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:24.185547 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:24.186042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:24.187112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:24.188272 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] RBAC: Adding query filter params (name=compute) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:24.188818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:24.191633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:24.203112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 359/710] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:24.204310 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:24.228312 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75606fc-5a5b-4b83-9d1d-8b8cc97a3b43 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:24.268430 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:24.268820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:24.269107 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:24.269384 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:24.270236 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] RBAC: Adding query filter params (role.id=8f11efc95ebc40fe9eb7752b60fab42f, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=6f52fc5710c944b1a0ef03df0b436271) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:24.270480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:24.272189 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ce0e-5de5-4f7b-8d4f-3abe322569dd None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:24.287606 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 352/711] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:24 2022] GET /identity/v3/role_assignments?user.id=6f52fc5710c944b1a0ef03df0b436271&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=8f11efc95ebc40fe9eb7752b60fab42f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:24.303756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:24.332606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab4542-4b85-4f4a-bbcf-206e21bbe684 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:24.375517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:24.376540 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:24.377057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:24.377540 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:24.378432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] RBAC: Adding query filter params (type=compute) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:24.378981 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:24.380668 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258661f2-7d7b-40d5-acae-dadde0675a34 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:24.386877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 360/712] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:25.666424 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec6ea7-2573-49a4-b589-33864c940da8 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:25.667043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec6ea7-2573-49a4-b589-33864c940da8 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:25.667043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec6ea7-2573-49a4-b589-33864c940da8 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:25.669115 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 353/713] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:25.674510 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b205-0526-479e-85d6-c8ffeba5239e None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:25.674907 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b205-0526-479e-85d6-c8ffeba5239e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:25.675096 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b205-0526-479e-85d6-c8ffeba5239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:25.693673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-1ad8b205-0526-479e-85d6-c8ffeba5239e None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:25.694861 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-1ad8b205-0526-479e-85d6-c8ffeba5239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:25.766174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 361/714] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:26.038385 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec6ea7-2573-49a4-b589-33864c940da8 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.129518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:26.137856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c76046-ef34-4783-b574-4541368cb73c None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.138147 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c76046-ef34-4783-b574-4541368cb73c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.138424 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c76046-ef34-4783-b574-4541368cb73c None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.140204 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 362/715] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:26.145470 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.145745 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.146163 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.146586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec6ea7-2573-49a4-b589-33864c940da8 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:26.167237 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:26.169124 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:26.182346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:26.183126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.183439 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.183742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.184823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:26.186184 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:26.195998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:26.197084 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:26.197993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 354/716] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:26.203693 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.241691 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:26.267610 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 363/717] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:26.330593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785588c7-9a6e-4b73-a647-e3002ccce878 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:26.364940 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:26.365551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.365825 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.366096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.366718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:26.367014 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:26.368410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b6467-ec45-4a41-a55a-ad104b9036b1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:26.373998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 355/718] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:26.381044 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.497973 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bb185-5f66-42f6-9e29-df8a78fc5e37 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:26.529319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef50ca-93f4-4534-8f73-6b79e3362f37 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.529656 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef50ca-93f4-4534-8f73-6b79e3362f37 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.529866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef50ca-93f4-4534-8f73-6b79e3362f37 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.531371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 356/719] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:26.536677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.537049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:26.537467 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.537652 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.537930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.538197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.538588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.538825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:26.539127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:26.540520 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:26.547018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 364/720] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Oct 17 06:01:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:26.555230 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.564967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:26.566672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:26.579128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:26.618796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:26.642616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 357/721] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:26.645475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.672594 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede1a3e-0658-4e67-b787-0a236bd92dc3 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:26.708515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a68da1-61aa-47f0-8162-997c3ae94869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:26.709182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68da1-61aa-47f0-8162-997c3ae94869 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.709397 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68da1-61aa-47f0-8162-997c3ae94869 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.709676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68da1-61aa-47f0-8162-997c3ae94869 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.710348 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a68da1-61aa-47f0-8162-997c3ae94869 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:26.711768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a68da1-61aa-47f0-8162-997c3ae94869 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:26.759763 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc1e37-eb73-42e5-9fcb-b12e33b101c6 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:26.793149 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 365/722] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:26.795157 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4345f-fef3-49aa-85f3-85c6c1d228a6 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.795543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4345f-fef3-49aa-85f3-85c6c1d228a6 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.795890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4345f-fef3-49aa-85f3-85c6c1d228a6 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.799355 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 366/723] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:26.803369 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.804062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.804532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.810654 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:26.811335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:26.811629 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:26.811913 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:26.812508 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:26.815345 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:26.826278 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:26.827547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:26.828480 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 358/724] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:26.830535 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:26.832038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:26.833648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.837756 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:26.931072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 367/725] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:26.932678 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:26.962407 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991e331-df19-4bb8-89f6-91e53105ddc2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:26.988034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:27.014171 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.014728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.015010 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.015288 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.016016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:27.016324 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.017830 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.024750 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 359/726] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:27.031118 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:27.063738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6dd20-4c36-4ee8-8487-027faeb1d77d None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:27.109740 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.111236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.111782 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.112519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] PATH_INFO: `/v3/services/image` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.113313 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.115757 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:27.116107 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:27.127600 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:27.128726 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:27.129725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:27.132921 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 368/727] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 06:01:26 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:27.138744 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:27.155699 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc6d62-6e2c-4cb8-95ad-a7a530d22dea None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:27.198872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.200410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.200973 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.201254 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.201898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:27.202205 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.203783 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.210363 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 360/728] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:27.217251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:27.252719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5221d-0878-4b40-9775-cfed78b1b5ba None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:27.255337 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:27.291651 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.292293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.292578 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.292854 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.293517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] RBAC: Adding query filter params (name=image) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:27.293823 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.295371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.303347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 369/729] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:27.303347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:27.325920 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7be80-2eaf-4858-ba74-464c1a1c7896 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:27.356600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:27.361823 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.362435 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.362711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.362981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.363667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.365138 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.414110 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06366f6d-22e1-4615-8364-66aca52c9dd4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:27.455700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 361/730] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:27.456805 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:27.578126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab87d07-0e02-42a8-8c18-ef4d935e307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.578754 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab87d07-0e02-42a8-8c18-ef4d935e307b None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.579038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab87d07-0e02-42a8-8c18-ef4d935e307b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.579331 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab87d07-0e02-42a8-8c18-ef4d935e307b None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.580018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab87d07-0e02-42a8-8c18-ef4d935e307b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.581591 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab87d07-0e02-42a8-8c18-ef4d935e307b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.592752 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1447d-9602-4f18-ba6a-54235b2229d8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:27.631980 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:27.632602 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:27.632879 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:27.633146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:27.633814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] RBAC: Adding query filter params (type=image) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:27.634114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:27.635597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd6373-869e-4302-8f18-62d9fb3bae80 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:27.641732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 362/731] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:27.695378 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 370/732] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:27 2022] POST /identity/v3/services => generated 242 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:29.076681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6080-ce16-40cd-be79-269f71f85aef None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.077238 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6080-ce16-40cd-be79-269f71f85aef None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.077238 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6080-ce16-40cd-be79-269f71f85aef None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.078814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 363/733] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:29.084227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.084637 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.084826 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.103640 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:29.104828 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:29.122983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:29.185606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 371/734] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:29.454343 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f6080-ce16-40cd-be79-269f71f85aef None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:29.510813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:29.561247 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f6080-ce16-40cd-be79-269f71f85aef None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:29.597662 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:29.598277 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.598558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.598833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.599520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:29.600949 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:29.603592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: WARNING keystone.server.flask.common [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 06:01:29.676323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.sql.core [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 06:01:29.676323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:01:29.676323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [parameters: {'id': 'b2526b6f16e648a2be188ddb2cf7f92d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 06:01:29.676323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-8d413666-f64e-4003-bb68-0c636f74c4be None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b2526b6f16e648a2be188ddb2cf7f92d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b2526b6f16e648a2be188ddb2cf7f92d. Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:01:29.694019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result.read() Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 06:01:29.694985 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application yield resource Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise self.value Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application session.commit() Oct 17 06:01:29.696064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:01:29.697074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:01:29.698075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result.read() Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:01:29.698955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [parameters: {'id': 'b2526b6f16e648a2be188ddb2cf7f92d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:29.702282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:29.704273 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b2526b6f16e648a2be188ddb2cf7f92d. Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:29.714975 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 364/735] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Oct 17 06:01:29 2022] POST /identity/v3/projects => generated 262 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:29.716512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:29.787628 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b540-3501-4439-b136-c07e5bdc1d16 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.787981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b540-3501-4439-b136-c07e5bdc1d16 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.788184 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b540-3501-4439-b136-c07e5bdc1d16 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.794145 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 365/736] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:29.800983 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55203007-6927-4e2e-aaa3-357d686cb011 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.801380 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55203007-6927-4e2e-aaa3-357d686cb011 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.801760 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55203007-6927-4e2e-aaa3-357d686cb011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.820985 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705edbf-a735-4aef-8197-6cbaf3a0671a None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:29.821522 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-55203007-6927-4e2e-aaa3-357d686cb011 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:29.823198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-55203007-6927-4e2e-aaa3-357d686cb011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:29.860795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:29.861462 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.861739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.862011 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.870421 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:29.872618 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-822b3e34-fb0c-4d96-84a8-627681eff6a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:29.885748 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:29.886799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:29.888426 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 372/737] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:29.889493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab3c0-1243-402b-8337-5867fa85335c None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.889493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab3c0-1243-402b-8337-5867fa85335c None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.889493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab3c0-1243-402b-8337-5867fa85335c None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.890765 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 373/738] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:29.895561 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab3c0-1243-402b-8337-5867fa85335c None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:29.907873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 366/739] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:29.907873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7bf86-bd13-4bcb-8b64-aee79734fc43 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:29.907873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7bf86-bd13-4bcb-8b64-aee79734fc43 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:29.907873 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7bf86-bd13-4bcb-8b64-aee79734fc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:29.911625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:29.927978 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-0da7bf86-bd13-4bcb-8b64-aee79734fc43 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:29.929379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-0da7bf86-bd13-4bcb-8b64-aee79734fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:29.963596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:30.005382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 367/740] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:30.006883 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019fd6-c632-4c02-9941-a018db8fe595 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.007166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019fd6-c632-4c02-9941-a018db8fe595 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.007468 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019fd6-c632-4c02-9941-a018db8fe595 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.008842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 368/741] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:30.014642 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.014930 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.015201 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.019547 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab3c0-1243-402b-8337-5867fa85335c None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.033971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:30.035150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:30.057871 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.058896 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.060041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.060647 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.061572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:30.062094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.064038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.087822 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 374/742] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:30.116534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 369/743] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:30.191109 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.273606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:30.298967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.306655 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53e26b-1614-4c13-bdb1-33d1cec2d6e9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.353296 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.354236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.354521 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.354795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.357137 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.358082 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:30.369226 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:30.370353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:30.371277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:30.373306 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 375/744] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:30.379128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.413680 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:30.417582 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ebb4-e8be-4cee-a30d-e41a9599f9c8 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.454882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.456226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.456588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.456867 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.457532 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.459021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.489206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e842d2-5ab5-4ca2-8e0f-99272495356b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.522189 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.522805 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.523088 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.523413 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.524098 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:30.524350 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.525732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.526796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:30.535491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 376/745] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:30.535491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.547346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 370/746] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:30 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:01:30.547346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.677177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dfc54-92e6-4614-9977-276867baedc7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.694912 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ab2ff-3ef5-452c-af80-361d3cd9b9c9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.718494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.719491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.719956 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.720443 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.721269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:30.721788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.723424 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.736855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 371/747] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:30.738027 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.740009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.741016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.741228 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.742489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] PATH_INFO: `/v3/services/compute` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.743103 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.744617 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:30.758558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:30.759792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:30.760847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:30.762804 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:30.763844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 377/748] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:30.769649 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.842538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:30.882197 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fbf2c-9adf-4506-a9ea-3df4d004d134 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.883561 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd7b4-e081-4dc1-99b9-0a26f8d14893 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:30.919904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.920831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.921328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.921804 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.922897 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] RBAC: Adding query filter params (name=compute) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:30.923542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.925230 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:30.925373 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.925849 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:30.926127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:30.926400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:30.927113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] RBAC: Adding query filter params (interface=admin, service_id=499fcfce81c94cc3a53c9b269b3769b1, region_id=RegionOne) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:30.927748 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:30.929557 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:30.939329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 372/750] 10.222.0.13 () {56 vars in 1249 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/endpoints?service_id=499fcfce81c94cc3a53c9b269b3769b1&interface=admin®ion_id=RegionOne => generated 193 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:30.939329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:30.939625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 378/750] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:30.944560 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:31.043698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:31.052779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8736ff2-643a-4d83-a894-cb4030343b7c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:31.056527 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629674d-7c1a-43e6-9773-f2e605a1f7f4 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:31.081070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:31.092638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:31.099338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:31.099338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:31.099338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:31.099338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:31.099338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:31.104285 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:31.104892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:31.105173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:31.105448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:31.106128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:31.106433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:31.106544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 379/751] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 06:01:31.107885 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:31.113842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 373/752] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:31.119780 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:31.231971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8a774-b215-4e01-8c68-508eac06e19b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:31.270528 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:31.271156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:31.271519 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:31.271797 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:31.272517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] RBAC: Adding query filter params (interface=public, service_id=6ca9a43816744fa8bebefb8fd41e4c45, region_id=RegionOne) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:31.272826 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:31.274212 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf66e3-2f5f-49f9-9d65-3baec3001641 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:31.280490 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 380/753] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:31 2022] GET /identity/v3/endpoints?service_id=6ca9a43816744fa8bebefb8fd41e4c45&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:31.288444 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:31.399042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c34b5-7ac1-4f4b-8217-4a4f1e06bda9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:31.436932 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e8a90-1985-42b5-baaa-cba4d0dd0764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:31.437947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8a90-1985-42b5-baaa-cba4d0dd0764 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:31.438449 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8a90-1985-42b5-baaa-cba4d0dd0764 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:31.438927 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8a90-1985-42b5-baaa-cba4d0dd0764 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:31.439774 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e8a90-1985-42b5-baaa-cba4d0dd0764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:31.441334 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e8a90-1985-42b5-baaa-cba4d0dd0764 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:31.448973 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 374/754] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:31 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:01:32.402026 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed80bb-6e03-4f00-9029-7544599ad886 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:32.403141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed80bb-6e03-4f00-9029-7544599ad886 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:32.403141 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed80bb-6e03-4f00-9029-7544599ad886 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:32.404821 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 381/755] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:32.410413 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753798-23a1-4149-bb44-e8b2064a4626 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:32.410838 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753798-23a1-4149-bb44-e8b2064a4626 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:32.410971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753798-23a1-4149-bb44-e8b2064a4626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:32.429474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-e1753798-23a1-4149-bb44-e8b2064a4626 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:32.430684 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-e1753798-23a1-4149-bb44-e8b2064a4626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:32.434407 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:32.508909 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 375/756] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:32.784059 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed80bb-6e03-4f00-9029-7544599ad886 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:32.794552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b15eac-9b4d-4954-a54d-5557fc970c82 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:32.794854 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b15eac-9b4d-4954-a54d-5557fc970c82 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:32.795110 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b15eac-9b4d-4954-a54d-5557fc970c82 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:32.796474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 376/757] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:32.801441 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:32.801717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:32.801995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:32.821210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:32.822496 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:32.846554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:32.898132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 377/758] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:32.906004 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed80bb-6e03-4f00-9029-7544599ad886 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:32.942707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:32.943700 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:32.944199 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:32.944684 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:32.945561 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:32.947215 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:32.950045 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: WARNING keystone.server.flask.common [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 06:01:32.996032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.sql.core [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 06:01:32.996032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 06:01:32.996032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [parameters: {'user_id': 'b565085cd1dc4774a3bf922c2d236ec1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 06:01:32.996032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218501) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result.read() Oct 17 06:01:33.004495 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:01:33.010696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application yield resource Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise self.value Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application session.commit() Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:01:33.011840 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:01:33.013323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:01:33.014453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application result.read() Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:01:33.015504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b565085cd1dc4774a3bf922c2d236ec1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:33.016480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 06:01:33.017361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 06:01:33.018511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 06:01:33.018511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 06:01:33.018511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:01:33.018511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:01:33.018511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 06:01:33.018511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.018935 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 382/759] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Oct 17 06:01:32 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:33.022475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.096534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:33.127601 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809f41c-dd53-4ce5-9be3-c1fb63299cfe None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:33.164420 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:33.165067 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.165348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.165641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.173039 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:33.174505 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:33.186759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:33.187892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 06:01:33.189300 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:33.189957 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.190052 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 378/760] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:33.200760 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.321266 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f35ec2-f3a1-4346-9f51-1a414dd48cae None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:33.331919 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414d28-8de3-4ebd-8e00-eaba1a9a563e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:33.341758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:33.362758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:33.363664 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.363954 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.364650 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] PATH_INFO: `/v3/services/image` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.364970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:33.366709 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:33.375277 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:33.376196 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:33.377242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:33.378301 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:33.379123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 383/761] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:33.385393 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.389384 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:33.390082 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.390423 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.447698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.447698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:33.447698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:33.447698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf08f-2ee8-4994-845d-d418b3cd6714 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:33.447698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 379/762] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:33.447698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e910ec-14f2-4eb5-9265-c91efa9d0d8d None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.448884 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e910ec-14f2-4eb5-9265-c91efa9d0d8d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.448884 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e910ec-14f2-4eb5-9265-c91efa9d0d8d None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.448884 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 380/763] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:33.453729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8af32-f25e-44fe-b62d-5da87256889b None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.454007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8af32-f25e-44fe-b62d-5da87256889b None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.454280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8af32-f25e-44fe-b62d-5da87256889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.485551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-1fe8af32-f25e-44fe-b62d-5da87256889b None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:33.493541 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-1fe8af32-f25e-44fe-b62d-5da87256889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:33.518988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c47a9-f85e-45cf-99d3-30570e725bf0 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:33.567523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:33.568364 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.568752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.569128 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.569904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] RBAC: Adding query filter params (name=image) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:33.570387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:33.572310 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:33.587405 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 384/764] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:33.589295 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.591948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 381/765] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:33 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:33.648217 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:33.695362 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437633dd-3af2-418d-ac6c-f8fae4b1463e None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.695740 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437633dd-3af2-418d-ac6c-f8fae4b1463e None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.696093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437633dd-3af2-418d-ac6c-f8fae4b1463e None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.697883 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 382/766] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:33.703166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.703478 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.703801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.708018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf766b-e7a5-4597-a4d2-309deb4aa1bb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:33.728806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:33.731687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:33.756661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:33.756661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.756969 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.757184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.757820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] RBAC: Adding query filter params (type=image) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:33.758123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:33.760581 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:33.767846 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 385/767] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:33.773884 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.822494 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 383/768] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:33 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:33.864583 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.878779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bced3b-e1ae-45b2-9c44-0e7bc661f16b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:33.888837 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:33.913351 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:33.913880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:33.914197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:33.914514 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:33.915219 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] RBAC: Adding query filter params (interface=public, service_id=242ca2d5363d484e8f26ddadae2cb902, region_id=RegionOne) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:33.915562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:33.916917 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:33.919589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:33.930846 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 386/769] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/endpoints?service_id=242ca2d5363d484e8f26ddadae2cb902&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:33.932496 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:33.972737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f395199-2ea0-4252-8431-2856f86662d2 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.007146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.007820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.008101 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.008377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.008959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.010393 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:34.019866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:34.020948 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:34.022001 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 384/770] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:34.026747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.036960 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3043b-d10f-49b2-9cab-fbfea0c0e327 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.070561 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.071173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.071487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.071763 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.072369 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.073741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.080843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 387/771] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:01:33 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:01:34.090918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.131700 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:34.138679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdea259-f35e-4dda-a124-12ffa88e2ea2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.154707 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:34.177839 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.178388 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.178665 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.178933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.179620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:34.179921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.181302 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.187717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 385/772] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:34.193325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.210200 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d4efd-6f10-4eaa-8a26-3dfaaf60334a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.251931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.252663 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.253028 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.253360 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] PATH_INFO: `/v3/services/compute` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.254005 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.255714 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:34.266022 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:34.267342 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:34.268468 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 388/773] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:34.273929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.312161 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52e5f-bb15-4ae8-941e-a7c31f7fcc32 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.351155 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.351809 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.352096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.352371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.353007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:34.353311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.354812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.360913 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 386/774] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Oct 17 06:01:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:34.366219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.381175 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:34.385407 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a6c22-d5ac-40bc-93c6-a3ddbcee99cb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.411655 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:34.424329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.424964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.425234 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.425503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.426148 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] RBAC: Adding query filter params (name=compute) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:34.426452 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.427920 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.433527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 389/775] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:34.439385 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.480054 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32c849-cfac-4093-9d4f-2812503b8ddc None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.516850 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e14a1c-d804-42ef-90ee-9abfe2ee4b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.517458 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14a1c-d804-42ef-90ee-9abfe2ee4b96 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.517737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14a1c-d804-42ef-90ee-9abfe2ee4b96 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.518006 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14a1c-d804-42ef-90ee-9abfe2ee4b96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.518667 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e14a1c-d804-42ef-90ee-9abfe2ee4b96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.520057 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e14a1c-d804-42ef-90ee-9abfe2ee4b96 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.545951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aefe65-8da7-4121-a283-929f6d490e1e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.580415 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.581026 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.581308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.581583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.582217 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:34.582520 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.583923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.594158 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 390/776] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Oct 17 06:01:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:34.594355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 387/777] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:01:34.596447 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:34.631635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:34.710245 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc767b52-bd90-4185-bbf4-033012c800d6 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:34.754598 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdef9a5-5d4c-407e-a8d0-c9d8f1725ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:34.755211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdef9a5-5d4c-407e-a8d0-c9d8f1725ff6 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:34.755518 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdef9a5-5d4c-407e-a8d0-c9d8f1725ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:34.755786 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdef9a5-5d4c-407e-a8d0-c9d8f1725ff6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:34.756447 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdef9a5-5d4c-407e-a8d0-c9d8f1725ff6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:34.757796 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdef9a5-5d4c-407e-a8d0-c9d8f1725ff6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:34.837702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 391/778] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:36.245010 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aac48c-f1e1-4c55-81d4-9a5579d259ee None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:36.245587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aac48c-f1e1-4c55-81d4-9a5579d259ee None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:36.245587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aac48c-f1e1-4c55-81d4-9a5579d259ee None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:36.247570 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 388/779] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:36.252740 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dfd48-048f-4a41-8b62-075155390648 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:36.253135 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dfd48-048f-4a41-8b62-075155390648 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:36.253304 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dfd48-048f-4a41-8b62-075155390648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:36.271324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-fa1dfd48-048f-4a41-8b62-075155390648 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:36.272828 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-fa1dfd48-048f-4a41-8b62-075155390648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:36.342722 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 392/780] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:36.609821 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aac48c-f1e1-4c55-81d4-9a5579d259ee None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:36.751798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:36.756747 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aac48c-f1e1-4c55-81d4-9a5579d259ee None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:36.791730 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:36.792369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:36.792647 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:36.792974 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] PATH_INFO: `/v3/services/image` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:36.793578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:36.794998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:36.804009 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:36.805167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 06:01:36.806141 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:36.808963 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eff969-6f5b-4b11-9caa-6f5768829391 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:36.809256 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eff969-6f5b-4b11-9caa-6f5768829391 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:36.809532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eff969-6f5b-4b11-9caa-6f5768829391 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:36.810410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 389/781] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Oct 17 06:01:36 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:36.810987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 393/782] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:36.811235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:36.816288 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:36.816569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:36.816839 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:36.830101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:36.835562 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:36.846838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:36.911536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 394/783] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:36.920634 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c302ba4-dd12-46ce-a714-82ed8d814f75 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:36.954935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:36.955608 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:36.955916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:36.956192 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:36.956850 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:36.957161 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:36.958578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2541f-a165-4e2b-bd0f-d4cdc222d4d7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:36.969857 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 390/784] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:36.971038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.072277 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5bb4d-522a-496f-bf33-ea9920cc5121 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.079534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53dc0e-b5ba-45e7-8cd0-1a538a60f8de None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.079829 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53dc0e-b5ba-45e7-8cd0-1a538a60f8de None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.080148 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53dc0e-b5ba-45e7-8cd0-1a538a60f8de None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.081572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 391/785] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:37.086371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.086660 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.086976 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.106413 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:37.107658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:37.108027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.108645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.108929 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.109206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.109851 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] RBAC: Adding query filter params (type=image) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:37.110172 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:37.111579 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:37.113640 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:37.122612 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 395/786] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:37.123801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.137521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:37.197697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 392/787] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:37.197697 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.236400 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ef3d1-9ad0-4aa9-abb7-76c5bbea6efd None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.273120 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.274696 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.274981 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.275258 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.275974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:37.277368 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:37.296053 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d90470-34de-4378-982c-68ff42c41752 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.330680 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.331340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.331625 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.331899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.333046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:37.334458 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:37.336056 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 393/788] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:37.341812 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.367981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:37.391984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 396/789] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:01:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:01:37.455978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.460150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8522b8c-b2e1-45ff-9926-db39fa244a35 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.514484 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.515142 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.515428 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.515707 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.516406 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:37.517871 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:37.530144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:37.577947 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0c743-5a4f-478d-8749-ddf9d6d5f846 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.614249 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.614894 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.615181 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.615531 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.620903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 394/790] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:37 2022] POST /identity/v3/projects => generated 311 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:01:37.627441 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:37.634978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:37.636396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:37.637524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 397/791] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:01:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:37.644994 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.731054 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:37.759790 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dab29-394f-4dd2-a075-bb149a380db3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.800653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.801020 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.801246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.801519 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.802445 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:37.802562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:37.804196 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:37.814339 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 395/792] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 06:01:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:37.820227 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:37.927585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec54c-19ad-4a40-b001-69c1005935be None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:37.944200 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:37.962211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:37.962851 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:37.963132 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:37.963437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:37.974511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:37.975983 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:37.991739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:37.992779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:37.993792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:37.993792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:37.993792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:37.993792 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 398/793] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:37.998959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:38.098784 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a72ee-1026-4563-8b43-b64ddefd26d9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:38.132690 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:38.133305 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:38.133587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:38.133861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:38.134510 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:38.134814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:38.136227 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:38.138828 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:38.154322 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 396/794] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:38.155606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:38.262939 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e054-6c10-49f9-9555-3a897e8dfd86 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:38.297517 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:38.298174 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:38.298460 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:38.298739 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:38.307348 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:38.308920 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:38.321690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:38.322944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:38.324094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 399/795] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 06:01:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:38.329174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:38.434117 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d88a8-7d7b-4811-8486-b4bc6c94690d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:38.475892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:38.476523 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:38.476867 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:38.477145 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:38.477906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:38.478218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:38.479666 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:38.501646 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 397/796] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 06:01:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:38.503070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:38.535681 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:38.614184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44917e92-7b98-4781-9c45-1930139067fd None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:38.649117 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:38.649797 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:38.650079 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:38.650343 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:38.660464 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:38.661891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:38.676329 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:38.677392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:38.684086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:38.684086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:38.684086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:38.684086 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 400/797] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:38.685956 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:38.742403 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:38.786041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6609-7c79-46bd-9358-5a0d9ff0789c None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:38.820883 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:38.821496 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:38.821774 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:38.822044 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:38.822679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:38.822979 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:38.824967 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:38.840551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 398/798] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:38.842140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:38.929690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:38.949051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a0a3-9fb1-4661-99cd-a487af02011c None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:38.984286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:38.984925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:38.985213 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:38.985489 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:38.992274 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:38.993717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:39.004902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:39.005905 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:39.007483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 401/799] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:39.012444 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:39.112725 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966483-2df4-402b-bdfd-053a17a244e4 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:39.126241 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:39.146536 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:39.147153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:39.147476 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:39.147751 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:39.148447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:39.148757 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:39.150174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb223ce-b222-4e42-b7c3-38a4c742a0f6 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:39.163252 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 399/800] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:39.169072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:39.269760 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6ab77-2e69-4b10-b2fc-a2027a7e2738 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:39.303631 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da7d31-0b48-49cb-9d30-e7220ae04042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:39.304359 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da7d31-0b48-49cb-9d30-e7220ae04042 None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:39.304575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da7d31-0b48-49cb-9d30-e7220ae04042 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:39.304853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da7d31-0b48-49cb-9d30-e7220ae04042 None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/9dcc0e6649de439eb130a3b5f5bc1b91/roles/8f11efc95ebc40fe9eb7752b60fab42f` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:39.307268 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da7d31-0b48-49cb-9d30-e7220ae04042 None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=9dcc0e6649de439eb130a3b5f5bc1b91, role_id=8f11efc95ebc40fe9eb7752b60fab42f)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:39.308753 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da7d31-0b48-49cb-9d30-e7220ae04042 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:39.404048 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:39.409589 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 402/801] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:01:39 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/9dcc0e6649de439eb130a3b5f5bc1b91/roles/8f11efc95ebc40fe9eb7752b60fab42f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:01:39.609041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085d7cb-e026-4936-b685-8c5cce936847 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:39.609589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085d7cb-e026-4936-b685-8c5cce936847 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:39.609589 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085d7cb-e026-4936-b685-8c5cce936847 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:39.610937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 400/802] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:39.616733 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515e92-ec90-4a79-b18c-ec247f1bb2d7 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:39.617018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515e92-ec90-4a79-b18c-ec247f1bb2d7 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:39.617293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515e92-ec90-4a79-b18c-ec247f1bb2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:39.636232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-5f515e92-ec90-4a79-b18c-ec247f1bb2d7 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:39.637971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-5f515e92-ec90-4a79-b18c-ec247f1bb2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:39.707236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 403/803] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:39.882801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812c6e7-9e5e-4143-a5e3-e02ea0c86489 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:39.883144 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812c6e7-9e5e-4143-a5e3-e02ea0c86489 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:39.883372 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812c6e7-9e5e-4143-a5e3-e02ea0c86489 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:39.884842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 401/804] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:39.891240 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:39.891240 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:39.891240 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:39.908975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:39.910167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:39.976344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812c6e7-9e5e-4143-a5e3-e02ea0c86489 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:39.979225 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 404/805] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:40.077597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812c6e7-9e5e-4143-a5e3-e02ea0c86489 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:40.111160 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:40.111801 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:40.112079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:40.112346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:40.113008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:40.114351 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:40.149854 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.sql.core [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:01:40.149854 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:01:40.149854 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [parameters: {'id': '31a2102a3e524da2b69add707654a461', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 06:01:40.149854 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=218502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:01:40.157896 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result.read() Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 06:01:40.158836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application yield resource Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application next(self.gen) Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise self.value Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application session.commit() Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 06:01:40.160767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.session.flush() Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application compat.raise_( Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 06:01:40.161807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 06:01:40.162835 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application conn.query(q) Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application result.read() Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 06:01:40.163767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [parameters: {'id': '31a2102a3e524da2b69add707654a461', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 06:01:40.164797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:40.165739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:40.166759 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 402/806] 10.222.0.13 () {60 vars in 1137 bytes} [Mon Oct 17 06:01:39 2022] POST /identity/v3/domains => generated 190 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:40.168893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:40.228083 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:40.250537 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:40.279438 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2a3bd-eed9-471c-aef8-024be6a85c59 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:40.297732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:40.318194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:40.318955 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:40.319234 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:40.319554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:40.330933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:40.338283 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:40.346964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:40.348058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:40.349038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:40.349038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:40.349038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:40.349038 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 405/807] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:40.355134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:40.367224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d485ff1-9931-4b60-b067-ba9d4d6382c1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:40.404163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:40.405259 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:40.405542 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:40.405813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:40.406921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:40.408605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:40.410174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 403/808] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:40.415758 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:40.472768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912efcd-1389-45e5-bfac-621b9ddf268b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:40.486575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:40.508401 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:40.509008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:40.509319 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:40.509605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:40.510459 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:40.510937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:40.513202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f439-bbea-4f98-9f85-f42031ae9127 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:40.521534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f2211-e8e5-4bfe-aa5c-739c66612896 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:40.525039 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 406/809] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:40.529135 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:40.560652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c4467-1eb8-4de4-af1f-58e142d0d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:40.561298 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4467-1eb8-4de4-af1f-58e142d0d2ab None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:40.561580 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4467-1eb8-4de4-af1f-58e142d0d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:40.561852 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4467-1eb8-4de4-af1f-58e142d0d2ab None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:40.562524 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c4467-1eb8-4de4-af1f-58e142d0d2ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:40.564011 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c4467-1eb8-4de4-af1f-58e142d0d2ab None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:40.631694 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 404/810] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:01:40 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 06:01:41.582836 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8073-3f5b-43ff-9fe9-be39c87f84e4 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:41.583388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8073-3f5b-43ff-9fe9-be39c87f84e4 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:41.583388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8073-3f5b-43ff-9fe9-be39c87f84e4 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:41.590863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 407/811] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:41.590956 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:41.590956 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:41.591251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:41.609501 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:41.610683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:41.675217 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 405/812] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:41.941136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8073-3f5b-43ff-9fe9-be39c87f84e4 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.042524 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8073-3f5b-43ff-9fe9-be39c87f84e4 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:42.076682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:42.077307 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.077589 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.077863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.087974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:42.092768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:42.093784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:42.095058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 408/813] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Oct 17 06:01:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:42.099851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.188956 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:42.197548 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d03288-3ea1-4d54-a6da-95aebc678cff None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:42.232008 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:42.232683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.233043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.233321 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.234019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:42.234325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:42.235819 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:42.244287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 406/814] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Oct 17 06:01:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:42.250716 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.281312 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:42.347575 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21450-c23b-486e-9f6d-727364ed42e9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:42.380923 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:42.381548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.381825 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.382097 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.391956 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:42.393382 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:42.408170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:42.409357 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:42.410628 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:42.410628 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:42.410628 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:42.411455 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 409/815] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:42.414908 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.514636 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a960c-ec72-40f2-939e-f59db3c8e5b5 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:42.547352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:42.547961 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.548237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.548504 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.549137 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:42.549436 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:42.550779 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:42.562429 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 407/816] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:42.568160 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.651497 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:42.667648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59ca0b-9d9d-4a46-b6ed-4d1c721ab942 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:42.702033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:42.702668 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.702948 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.703219 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] PATH_INFO: `/v3/users/nova` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.711282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:42.712772 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-d9362e90-d77a-43cf-ac82-5840ff18776f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:01:42.726231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:42.727465 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:42.729087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 410/817] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Oct 17 06:01:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:42.734050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.750106 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:42.830967 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15acf4f-b07b-43ec-8a34-24d2d2f3ccb0 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.831248 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15acf4f-b07b-43ec-8a34-24d2d2f3ccb0 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.831553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15acf4f-b07b-43ec-8a34-24d2d2f3ccb0 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.832883 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 411/818] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:42.834445 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d270-a1e8-43aa-a735-a63df8bf3f45 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:42.837814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.838098 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.838371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.856672 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:42.857858 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:42.868459 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:42.869068 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:42.869344 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:42.869613 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:42.870333 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:42.870634 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:42.872157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:42.889966 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 408/819] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Oct 17 06:01:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:42.895777 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:42.927052 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 412/820] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:42.963887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:42.992729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94df42-3cfd-4ac9-8e9e-86b28435e91b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.026094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.026695 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.026973 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.027861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.091803 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.093233 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.157486 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.186853 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 409/821] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 17 06:01:43.192738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.194587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.266471 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.291203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec485488-89b4-482d-8839-8b7475cb72c9 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.292604 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef0466-9c2f-4fc8-b189-49664dcd8f37 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.325569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.326198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.326429 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.326560 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.326737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.327055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.327361 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.327638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.336755 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.337863 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:43.338194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:43.342681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:43.343788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:01:43.345202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 410/822] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:43.349717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:01:43.354398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:43.355596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:01:43.357010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:01:43.357010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:01:43.357010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:43.357010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 413/823] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:43.360984 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.380694 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.448586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff733414-bda7-49ea-9cb6-dca4878c711b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.461282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fbe10-c0b2-42af-92f1-1a0d18463756 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.487327 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.488175 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.488789 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.489072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.489348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.490013 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:43.490317 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.491699 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.495752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.496895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.497183 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.497444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.498081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:43.498408 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.499777 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.501656 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 411/824] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:43.506218 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.517223 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 414/825] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:43.518130 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.600162 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.607541 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0407cf-9b4a-4430-af0b-aa9b4db2f3b3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.616027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638db9-bced-4fc5-93a9-d25e37269813 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.641436 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.642069 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.642339 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.642615 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] PATH_INFO: `/v3/users/867967cd1728462a9a83155146be2b8c` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.651232 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.651925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.652483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.652483 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.655918 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] RBAC: Authorizing `identity:get_user(user_id=867967cd1728462a9a83155146be2b8c)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.657346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.658964 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 412/826] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/users/867967cd1728462a9a83155146be2b8c => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:43.659201 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.660638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.664774 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:01:43.674354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:43.675485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:43.676499 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 415/827] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:01:43.681318 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.726717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.761737 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b53ede-0e52-4761-96cb-9107838287ea None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.780145 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4eb6-4be9-49b9-a966-baffbe71897a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.794723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.795382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.795657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.795928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] PATH_INFO: `/v3/projects/43b44472fc1f41e3b0b769f3561e2a93` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.797028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] RBAC: Authorizing `identity:get_project(project_id=43b44472fc1f41e3b0b769f3561e2a93)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.798433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.800075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 413/828] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/projects/43b44472fc1f41e3b0b769f3561e2a93 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:43.805866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.815620 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.816228 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.816512 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.816781 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.817475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:43.817779 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.819187 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.828226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.831072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 416/829] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:01:43.837191 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:43.904661 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8d67-6686-4f18-b04f-b09f95d5efa1 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.937204 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e910b3-7aa8-4b55-a7e6-42d84e3f1156 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:43.940863 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da620da-8ca3-4678-ad65-037acefc90b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.941530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da620da-8ca3-4678-ad65-037acefc90b8 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.941814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da620da-8ca3-4678-ad65-037acefc90b8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.942093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da620da-8ca3-4678-ad65-037acefc90b8 None admin] PATH_INFO: `/v3/projects/43b44472fc1f41e3b0b769f3561e2a93/users/867967cd1728462a9a83155146be2b8c/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.944051 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da620da-8ca3-4678-ad65-037acefc90b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=43b44472fc1f41e3b0b769f3561e2a93, user_id=867967cd1728462a9a83155146be2b8c, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.944680 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:43.945507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da620da-8ca3-4678-ad65-037acefc90b8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.971058 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:43.971782 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:43.976570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:43.976570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:43.976570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] RBAC: Adding query filter params (role.id=8f11efc95ebc40fe9eb7752b60fab42f, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=9dcc0e6649de439eb130a3b5f5bc1b91) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:43.976570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:43.976570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860e256-c242-4658-ad88-7c2d758aa2ec None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:43.981952 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 417/830] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:43 2022] GET /identity/v3/role_assignments?user.id=9dcc0e6649de439eb130a3b5f5bc1b91&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=8f11efc95ebc40fe9eb7752b60fab42f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:44.031798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 414/831] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:01:43 2022] PUT /identity/v3/projects/43b44472fc1f41e3b0b769f3561e2a93/users/867967cd1728462a9a83155146be2b8c/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 06:01:46.167916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb2eb0-fdfc-4ed8-8cb1-709071d86ed4 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:46.168482 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb2eb0-fdfc-4ed8-8cb1-709071d86ed4 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:46.168482 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb2eb0-fdfc-4ed8-8cb1-709071d86ed4 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:46.169813 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 418/832] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:46.174976 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:46.175442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:46.175549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:46.194043 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:46.195223 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:46.259131 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 415/833] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:46.528325 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb2eb0-fdfc-4ed8-8cb1-709071d86ed4 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:46.632015 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb2eb0-fdfc-4ed8-8cb1-709071d86ed4 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:46.731794 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:46.732347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:46.732629 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:46.732904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:46.743133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:01:46.748276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:01:46.751702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:01:46.752880 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 419/834] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:01:46.755441 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:46.767311 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:46.853335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c20fb-77d5-4995-a2cd-1d928e1b0066 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:46.885842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:46.887007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:46.887290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:46.887592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:46.888248 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:46.888550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:46.889906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:46.897632 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 416/835] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:46.903714 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:46.934243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:47.000992 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa6c69-d8bd-4586-be3a-e51cb72d5a46 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:47.034032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:47.034648 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:47.034930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:47.035207 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] PATH_INFO: `/v3/users/867967cd1728462a9a83155146be2b8c` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:47.036353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] RBAC: Authorizing `identity:get_user(user_id=867967cd1728462a9a83155146be2b8c)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:47.037772 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:47.039347 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 420/836] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:01:46 2022] GET /identity/v3/users/867967cd1728462a9a83155146be2b8c => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:47.045291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:47.113384 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:47.142849 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d908b-2819-4ff0-9910-1d0d59aaf442 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:47.174773 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:47.175397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:47.175935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:47.175935 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:47.240960 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:47.242395 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:47.307880 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:47.336423 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 417/837] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:01:47.342390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:47.437444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:47.441151 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d395919-87cf-45d1-9f00-3d338f1f104d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:47.473973 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8eea76-d529-440e-9399-391e2a010d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:47.474596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8eea76-d529-440e-9399-391e2a010d81 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:47.474873 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8eea76-d529-440e-9399-391e2a010d81 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:47.475142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8eea76-d529-440e-9399-391e2a010d81 None admin] PATH_INFO: `/v3/projects/43b44472fc1f41e3b0b769f3561e2a93` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:47.476312 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8eea76-d529-440e-9399-391e2a010d81 None admin] RBAC: Authorizing `identity:get_project(project_id=43b44472fc1f41e3b0b769f3561e2a93)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:47.477735 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8eea76-d529-440e-9399-391e2a010d81 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:47.479356 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 421/838] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:01:47 2022] GET /identity/v3/projects/43b44472fc1f41e3b0b769f3561e2a93 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:47.485974 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:47.582899 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d5bbd-0a29-4750-bd54-c7a0c2b6150b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:47.616588 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:47.617188 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:47.617469 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:47.617742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:47.618457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=43b44472fc1f41e3b0b769f3561e2a93, user.id=867967cd1728462a9a83155146be2b8c) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:47.618766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:47.620279 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3dc3a2-192d-4fd8-be00-dd1399fc2f53 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:47.626427 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 418/839] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:47 2022] GET /identity/v3/role_assignments?user.id=867967cd1728462a9a83155146be2b8c&scope.project.id=43b44472fc1f41e3b0b769f3561e2a93&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:49.752072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca088d-771b-4fd7-a41f-f2247ba59de7 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:49.752584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca088d-771b-4fd7-a41f-f2247ba59de7 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:49.752584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca088d-771b-4fd7-a41f-f2247ba59de7 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:49.753867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 422/840] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:01:49.759163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:49.759604 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:49.759736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:49.777517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:49.779189 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:49.807280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:49.843678 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 419/841] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:50.110760 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca088d-771b-4fd7-a41f-f2247ba59de7 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:50.208819 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca088d-771b-4fd7-a41f-f2247ba59de7 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:50.241412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f243d46-f9a1-4af4-93a0-53dc1e10ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:50.242027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243d46-f9a1-4af4-93a0-53dc1e10ed85 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:50.242307 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243d46-f9a1-4af4-93a0-53dc1e10ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:50.242578 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f243d46-f9a1-4af4-93a0-53dc1e10ed85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:50.243718 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243d46-f9a1-4af4-93a0-53dc1e10ed85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:50.245149 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f243d46-f9a1-4af4-93a0-53dc1e10ed85 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:50.246676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 423/842] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:50.252786 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:50.347400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813fc9c-3b0e-4b51-a835-df46834ceed0 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:50.379661 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed53b0-814b-4cf7-ba66-18b45536b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:50.380195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed53b0-814b-4cf7-ba66-18b45536b949 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:50.380471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed53b0-814b-4cf7-ba66-18b45536b949 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:50.380738 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed53b0-814b-4cf7-ba66-18b45536b949 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:50.381369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed53b0-814b-4cf7-ba66-18b45536b949 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:50.382767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed53b0-814b-4cf7-ba66-18b45536b949 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:50.487871 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 420/843] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:01:50 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:01:52.665341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46b9bb-43e7-4496-8124-caa3e6276843 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:52.665902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46b9bb-43e7-4496-8124-caa3e6276843 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:52.665902 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46b9bb-43e7-4496-8124-caa3e6276843 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:52.667375 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 424/844] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:52.673376 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:52.673796 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:52.674003 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:52.692302 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:52.693499 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:52.709363 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:52.758506 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 421/845] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 06:01:53.025969 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46b9bb-43e7-4496-8124-caa3e6276843 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:53.060549 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:53.125146 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46b9bb-43e7-4496-8124-caa3e6276843 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:53.158717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:53.159362 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:53.159647 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:53.159919 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] PATH_INFO: `/v3/roles/e5f3830b373b402693826265019b9344` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:53.161318 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] RBAC: Authorizing `identity:get_role(role_id=e5f3830b373b402693826265019b9344)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:53.162677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:53.164015 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 425/846] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:01:53 2022] GET /identity/v3/roles/e5f3830b373b402693826265019b9344 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:53.169814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:53.265509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55e845-d8c0-4949-8431-8a0b6da88dab None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:53.298362 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:53.298929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:53.299204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:53.299511 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] PATH_INFO: `/v3/users/d025f7c3b7a445faa3755c7cb398c93a` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:53.426866 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] RBAC: Authorizing `identity:get_user(user_id=d025f7c3b7a445faa3755c7cb398c93a)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:53.428327 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:53.429978 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 422/847] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:01:53 2022] GET /identity/v3/users/d025f7c3b7a445faa3755c7cb398c93a => generated 286 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:53.436554 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:53.534198 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211f3ab-ea74-4cc3-ac4c-75f8d509b41d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:53.535722 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:53.569636 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02f71-0b0f-4a9a-b49d-c325462971fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:53.570202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02f71-0b0f-4a9a-b49d-c325462971fa None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:53.570485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02f71-0b0f-4a9a-b49d-c325462971fa None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:53.570760 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02f71-0b0f-4a9a-b49d-c325462971fa None admin] PATH_INFO: `/v3/projects/43b44472fc1f41e3b0b769f3561e2a93` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:53.571918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad02f71-0b0f-4a9a-b49d-c325462971fa None admin] RBAC: Authorizing `identity:get_project(project_id=43b44472fc1f41e3b0b769f3561e2a93)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:53.573354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad02f71-0b0f-4a9a-b49d-c325462971fa None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:53.574972 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 426/848] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:01:53 2022] GET /identity/v3/projects/43b44472fc1f41e3b0b769f3561e2a93 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:53.581607 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:53.593783 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:53.677551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d06db-0db4-451f-a55c-7296cd1b1136 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:53.710495 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49864318-954f-4bd1-af4e-c3ff547e9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:53.711143 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49864318-954f-4bd1-af4e-c3ff547e9ae8 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:53.711453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49864318-954f-4bd1-af4e-c3ff547e9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:53.711729 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49864318-954f-4bd1-af4e-c3ff547e9ae8 None admin] PATH_INFO: `/v3/projects/43b44472fc1f41e3b0b769f3561e2a93/users/d025f7c3b7a445faa3755c7cb398c93a/roles/e5f3830b373b402693826265019b9344` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:53.713836 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49864318-954f-4bd1-af4e-c3ff547e9ae8 None admin] RBAC: Authorizing `identity:create_grant(project_id=43b44472fc1f41e3b0b769f3561e2a93, user_id=d025f7c3b7a445faa3755c7cb398c93a, role_id=e5f3830b373b402693826265019b9344)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:53.715246 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49864318-954f-4bd1-af4e-c3ff547e9ae8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:53.812927 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 423/849] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:01:53 2022] PUT /identity/v3/projects/43b44472fc1f41e3b0b769f3561e2a93/users/d025f7c3b7a445faa3755c7cb398c93a/roles/e5f3830b373b402693826265019b9344 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:01:55.911541 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa27ff-2b62-4d9c-ab90-ea5aeb965c24 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:55.912146 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa27ff-2b62-4d9c-ab90-ea5aeb965c24 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:55.912146 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa27ff-2b62-4d9c-ab90-ea5aeb965c24 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:55.913424 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 427/850] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:55.918579 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:55.919003 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:55.919149 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:55.937005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:55.938199 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:55.997710 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:56.002031 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 424/851] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:56.271390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa27ff-2b62-4d9c-ab90-ea5aeb965c24 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:56.367491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa27ff-2b62-4d9c-ab90-ea5aeb965c24 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:56.400231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:56.400845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:56.401123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:56.401396 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] PATH_INFO: `/v3/roles/e5f3830b373b402693826265019b9344` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:56.402775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] RBAC: Authorizing `identity:get_role(role_id=e5f3830b373b402693826265019b9344)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:56.404190 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:56.405475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 428/852] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:01:56 2022] GET /identity/v3/roles/e5f3830b373b402693826265019b9344 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:56.411037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:56.507100 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6c99f-0505-4f68-a423-868d5b9aa3ac None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:56.540150 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:56.540720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:56.540997 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:56.541267 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] PATH_INFO: `/v3/users/d025f7c3b7a445faa3755c7cb398c93a` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:56.542377 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d025f7c3b7a445faa3755c7cb398c93a)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:56.543828 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:56.545384 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 425/853] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:01:56 2022] GET /identity/v3/users/d025f7c3b7a445faa3755c7cb398c93a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:56.551202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:56.591702 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:56.650042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca15e3-e1be-4cc3-aecb-2ef1c3f6a648 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:56.683482 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:56.684074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:56.684891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:56.685160 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:56.750731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:56.752209 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:56.788374 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:56.846078 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 429/854] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:01:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:01:56.852059 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:56.916197 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:56.949203 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc539e0-6d22-46c5-a1de-7f86406b23b9 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:56.980592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf16d-46df-4174-838b-2704ca61d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:56.981843 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf16d-46df-4174-838b-2704ca61d1ad None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:56.982124 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf16d-46df-4174-838b-2704ca61d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:56.982404 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf16d-46df-4174-838b-2704ca61d1ad None admin] PATH_INFO: `/v3/projects/43b44472fc1f41e3b0b769f3561e2a93` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:56.983564 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf16d-46df-4174-838b-2704ca61d1ad None admin] RBAC: Authorizing `identity:get_project(project_id=43b44472fc1f41e3b0b769f3561e2a93)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:56.984972 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf16d-46df-4174-838b-2704ca61d1ad None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:56.986565 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 426/855] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:01:56 2022] GET /identity/v3/projects/43b44472fc1f41e3b0b769f3561e2a93 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:56.992649 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:57.089286 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c14d47-1147-46eb-a378-a6f62d55f97b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:57.122041 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:57.122569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:57.122853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:57.123122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:57.123936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] RBAC: Adding query filter params (role.id=e5f3830b373b402693826265019b9344, scope.project.id=43b44472fc1f41e3b0b769f3561e2a93, user.id=d025f7c3b7a445faa3755c7cb398c93a) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:01:57.124243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:57.125655 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a72c29-56d8-4264-a8db-8a4ae72565f6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:57.131178 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 430/856] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:01:56 2022] GET /identity/v3/role_assignments?user.id=d025f7c3b7a445faa3755c7cb398c93a&scope.project.id=43b44472fc1f41e3b0b769f3561e2a93&role.id=e5f3830b373b402693826265019b9344 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:59.255864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e48296-437b-4c2a-8488-87a039c17e07 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:59.256438 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e48296-437b-4c2a-8488-87a039c17e07 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:59.256438 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e48296-437b-4c2a-8488-87a039c17e07 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:59.257720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 427/857] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:01:59.263476 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:59.263868 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:59.264040 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:59.270412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:59.282538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:01:59.283774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:01:59.350076 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 431/858] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:01:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:01:59.624418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e48296-437b-4c2a-8488-87a039c17e07 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:59.721369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e48296-437b-4c2a-8488-87a039c17e07 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:59.754592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d929e-011b-47f0-89e1-a3e0d65682d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:59.755198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d929e-011b-47f0-89e1-a3e0d65682d7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:59.755497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d929e-011b-47f0-89e1-a3e0d65682d7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:59.755766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d929e-011b-47f0-89e1-a3e0d65682d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:59.756863 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d929e-011b-47f0-89e1-a3e0d65682d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:59.758261 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d929e-011b-47f0-89e1-a3e0d65682d7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:59.759805 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 428/859] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:01:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:01:59.765365 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:01:59.859858 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ccebd-11e2-403c-acb7-26cc0d7dc909 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:01:59.888116 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:01:59.892960 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41801072-708b-40a1-9509-e09cafbf56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:01:59.893552 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801072-708b-40a1-9509-e09cafbf56ec None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:01:59.901725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801072-708b-40a1-9509-e09cafbf56ec None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:01:59.901725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801072-708b-40a1-9509-e09cafbf56ec None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:01:59.901725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41801072-708b-40a1-9509-e09cafbf56ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:01:59.901725 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41801072-708b-40a1-9509-e09cafbf56ec None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:01:59.990371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 432/860] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:01:59 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:02:02.143495 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c05afb-3dd5-4ec0-bbca-a0c2d5621dda None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:02.144062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c05afb-3dd5-4ec0-bbca-a0c2d5621dda None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:02.144062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c05afb-3dd5-4ec0-bbca-a0c2d5621dda None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:02.145363 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 429/861] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:02.150543 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:02.151000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:02.151099 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:02.169225 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:02.170398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:02.234355 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 433/862] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:02.500158 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c05afb-3dd5-4ec0-bbca-a0c2d5621dda None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:02.504018 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:02.597074 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c05afb-3dd5-4ec0-bbca-a0c2d5621dda None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:02.629162 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ec39-8598-49cd-9fbf-8bc5b8f92c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:02.629771 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ec39-8598-49cd-9fbf-8bc5b8f92c3e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:02.630046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ec39-8598-49cd-9fbf-8bc5b8f92c3e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:02.630313 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ec39-8598-49cd-9fbf-8bc5b8f92c3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:02.631414 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ec39-8598-49cd-9fbf-8bc5b8f92c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:02.632814 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ec39-8598-49cd-9fbf-8bc5b8f92c3e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:02.634326 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 430/863] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:02.640503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:02.736165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d60556-5c3d-4455-8503-0437c31c7af1 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:02.749427 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:02.769009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877952-b252-4174-ae51-303d897d2d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:02.769611 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877952-b252-4174-ae51-303d897d2d7f None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:02.769895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877952-b252-4174-ae51-303d897d2d7f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:02.770169 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877952-b252-4174-ae51-303d897d2d7f None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:02.770823 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877952-b252-4174-ae51-303d897d2d7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:02.772275 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877952-b252-4174-ae51-303d897d2d7f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:02.875112 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 434/864] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:02:02 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:02:05.010161 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7da95-15f2-4620-9c52-233c0f429f16 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:05.010723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7da95-15f2-4620-9c52-233c0f429f16 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:05.010723 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7da95-15f2-4620-9c52-233c0f429f16 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:05.012558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 431/865] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:05.017872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:05.018267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:05.018436 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:05.037103 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:05.038309 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:05.102866 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 435/866] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:05.375050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7da95-15f2-4620-9c52-233c0f429f16 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:05.443447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:05.472641 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7da95-15f2-4620-9c52-233c0f429f16 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:05.505282 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:05.505903 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:05.506179 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:05.506446 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:05.516126 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:02:05.520905 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:02:05.522073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:02:05.523237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 432/867] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:02:05.528565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:05.623088 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:05.626010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5b515-3668-49f9-952e-6ceb58bc7246 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:05.659802 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:05.660404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:05.660688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:05.660964 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:05.661630 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:02:05.661937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:05.663381 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:05.671962 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 436/868] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:05.677770 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:05.790234 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da323f81-6077-41c1-9aab-b7ff54c10467 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:05.819373 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:05.824070 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:05.824718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:05.824997 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:05.825272 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] PATH_INFO: `/v3/users/81486cb68e2c4cb1bbefbcc0da3fbb39` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:05.838000 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] RBAC: Authorizing `identity:get_user(user_id=81486cb68e2c4cb1bbefbcc0da3fbb39)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:05.839457 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:05.841061 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 433/869] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:02:05 2022] GET /identity/v3/users/81486cb68e2c4cb1bbefbcc0da3fbb39 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:05.846824 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:05.945330 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e598-8ce9-4489-aafb-e5224f81e748 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:05.977033 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe741e50-ff73-46a7-8e55-1ec0c82b7e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:05.977645 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe741e50-ff73-46a7-8e55-1ec0c82b7e32 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:05.977922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe741e50-ff73-46a7-8e55-1ec0c82b7e32 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:05.978192 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe741e50-ff73-46a7-8e55-1ec0c82b7e32 None admin] PATH_INFO: `/v3/projects/b28a387457154174bb6aeb1974f7a94f` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:05.979292 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe741e50-ff73-46a7-8e55-1ec0c82b7e32 None admin] RBAC: Authorizing `identity:get_project(project_id=b28a387457154174bb6aeb1974f7a94f)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:05.981233 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe741e50-ff73-46a7-8e55-1ec0c82b7e32 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:05.982999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 437/870] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:02:05 2022] GET /identity/v3/projects/b28a387457154174bb6aeb1974f7a94f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:05.988530 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:06.084032 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dae84-6a86-41ed-a663-99b10a54f874 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:06.115874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7799d2-4420-45b1-93f6-4e44b55cbb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:06.116521 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7799d2-4420-45b1-93f6-4e44b55cbb04 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:06.116799 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7799d2-4420-45b1-93f6-4e44b55cbb04 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:06.117072 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7799d2-4420-45b1-93f6-4e44b55cbb04 None admin] PATH_INFO: `/v3/projects/b28a387457154174bb6aeb1974f7a94f/users/81486cb68e2c4cb1bbefbcc0da3fbb39/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:06.118965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7799d2-4420-45b1-93f6-4e44b55cbb04 None admin] RBAC: Authorizing `identity:create_grant(project_id=b28a387457154174bb6aeb1974f7a94f, user_id=81486cb68e2c4cb1bbefbcc0da3fbb39, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:06.120397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7799d2-4420-45b1-93f6-4e44b55cbb04 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:06.212392 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 434/871] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:02:05 2022] PUT /identity/v3/projects/b28a387457154174bb6aeb1974f7a94f/users/81486cb68e2c4cb1bbefbcc0da3fbb39/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:02:08.338403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ca45c-683b-42cf-a0a9-ebca46ad2efd None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:08.339055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ca45c-683b-42cf-a0a9-ebca46ad2efd None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:08.339055 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ca45c-683b-42cf-a0a9-ebca46ad2efd None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:08.340807 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 438/872] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:08.345970 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:08.346410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:08.346518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:08.364929 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:08.366111 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:08.411380 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:08.432412 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 435/873] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:08.701701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ca45c-683b-42cf-a0a9-ebca46ad2efd None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:08.732140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:08.801002 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ca45c-683b-42cf-a0a9-ebca46ad2efd None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:08.833915 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:08.834530 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:08.834806 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:08.835075 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:08.844961 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:02:08.849676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:02:08.850933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:02:08.852065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 439/874] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:02:08.857336 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:08.951515 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2e52-b68c-489b-b435-ec088bcdc9e1 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:08.984146 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:08.984681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:08.984960 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:08.985231 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:08.985887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:02:08.986189 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:08.987561 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:08.995170 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 436/875] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:09.000920 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:09.087553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:09.097570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e9675-dfa4-4cd1-80d6-89cdb8c8fbb6 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:09.131091 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:09.131737 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:09.132017 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:09.132291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] PATH_INFO: `/v3/users/81486cb68e2c4cb1bbefbcc0da3fbb39` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:09.133397 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] RBAC: Authorizing `identity:get_user(user_id=81486cb68e2c4cb1bbefbcc0da3fbb39)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:09.134814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:09.136390 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 440/876] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:02:08 2022] GET /identity/v3/users/81486cb68e2c4cb1bbefbcc0da3fbb39 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:09.142079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:09.184599 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:09.239221 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3d6db-61fe-4aea-83fa-6dafd7e692f0 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:09.271179 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:09.271808 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:09.272089 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:09.272360 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:09.337328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:09.338742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:09.378369 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:09.433633 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 437/877] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:02:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:02:09.440238 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:09.535738 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c3f3e-faf2-4998-9165-07e32f3ececb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:09.568841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aef60-ef86-4a8a-8194-6bb05875c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:09.569452 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aef60-ef86-4a8a-8194-6bb05875c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:09.569730 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aef60-ef86-4a8a-8194-6bb05875c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:09.569999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aef60-ef86-4a8a-8194-6bb05875c0b3 None admin] PATH_INFO: `/v3/projects/b28a387457154174bb6aeb1974f7a94f` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:09.571102 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aef60-ef86-4a8a-8194-6bb05875c0b3 None admin] RBAC: Authorizing `identity:get_project(project_id=b28a387457154174bb6aeb1974f7a94f)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:09.572551 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aef60-ef86-4a8a-8194-6bb05875c0b3 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:09.574146 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 441/878] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:02:09 2022] GET /identity/v3/projects/b28a387457154174bb6aeb1974f7a94f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:09.580290 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:09.676754 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1478f-2ffe-4eb8-8be5-fe36e86b0cda None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:09.709596 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:09.710193 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:09.710473 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:09.710767 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:09.711493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=b28a387457154174bb6aeb1974f7a94f, user.id=81486cb68e2c4cb1bbefbcc0da3fbb39) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:02:09.711804 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:09.713204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f70380-21a5-4ecd-8636-43d99f83ffac None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:09.718841 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 438/879] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:02:09 2022] GET /identity/v3/role_assignments?user.id=81486cb68e2c4cb1bbefbcc0da3fbb39&scope.project.id=b28a387457154174bb6aeb1974f7a94f&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:11.844536 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd88423-1a17-496c-801b-44fe6075c01e None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:11.845084 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd88423-1a17-496c-801b-44fe6075c01e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:11.845084 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd88423-1a17-496c-801b-44fe6075c01e None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:11.846408 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 442/880] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:11.851724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977961c4-4e05-431a-ad3d-8c59bfe93f3b None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:11.852233 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977961c4-4e05-431a-ad3d-8c59bfe93f3b None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:11.852337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977961c4-4e05-431a-ad3d-8c59bfe93f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:11.864833 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:11.870247 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-977961c4-4e05-431a-ad3d-8c59bfe93f3b None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:11.871462 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-977961c4-4e05-431a-ad3d-8c59bfe93f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:11.936035 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 439/881] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:12.200516 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd88423-1a17-496c-801b-44fe6075c01e None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:12.226841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:12.299522 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd88423-1a17-496c-801b-44fe6075c01e None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:12.333168 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424e758-bf69-4167-91ec-547e0e2dd803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:12.333762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424e758-bf69-4167-91ec-547e0e2dd803 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:12.334044 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424e758-bf69-4167-91ec-547e0e2dd803 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:12.334316 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424e758-bf69-4167-91ec-547e0e2dd803 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:12.334987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424e758-bf69-4167-91ec-547e0e2dd803 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:12.336387 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424e758-bf69-4167-91ec-547e0e2dd803 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:12.419487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 443/882] 10.222.0.13 () {60 vars in 1138 bytes} [Mon Oct 17 06:02:12 2022] POST /identity/v3/domains => generated 267 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:02:14.563864 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bd78b-15b7-4339-8842-76fefc51508f None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:14.564423 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bd78b-15b7-4339-8842-76fefc51508f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:14.564423 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bd78b-15b7-4339-8842-76fefc51508f None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:14.565728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 440/883] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:14.571021 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:14.571475 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:14.571608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:14.589451 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:14.591131 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:14.656243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 444/884] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:14.922033 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bd78b-15b7-4339-8842-76fefc51508f None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:15.018913 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bd78b-15b7-4339-8842-76fefc51508f None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:15.051017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cbe4b-248e-4130-8ed0-7aa14b51e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:15.051658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cbe4b-248e-4130-8ed0-7aa14b51e7eb None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:15.051937 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cbe4b-248e-4130-8ed0-7aa14b51e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:15.052204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cbe4b-248e-4130-8ed0-7aa14b51e7eb None admin] PATH_INFO: `/v3/domains/7dfe0ad7c4864d7e9a8a5280f4e1b193` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:15.054780 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:15.063205 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cbe4b-248e-4130-8ed0-7aa14b51e7eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=7dfe0ad7c4864d7e9a8a5280f4e1b193)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:15.064643 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cbe4b-248e-4130-8ed0-7aa14b51e7eb None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:15.066172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 441/885] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Oct 17 06:02:14 2022] GET /identity/v3/domains/7dfe0ad7c4864d7e9a8a5280f4e1b193 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:15.071970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:15.088999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:15.169772 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380961c-3c0c-4c54-ad7c-387f58e00f26 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:15.203159 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aa3dd-6659-4c07-a999-fe4a30b85673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:15.203734 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aa3dd-6659-4c07-a999-fe4a30b85673 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:15.204016 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aa3dd-6659-4c07-a999-fe4a30b85673 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:15.204287 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aa3dd-6659-4c07-a999-fe4a30b85673 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:15.204927 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543aa3dd-6659-4c07-a999-fe4a30b85673 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:15.206346 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543aa3dd-6659-4c07-a999-fe4a30b85673 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:15.299165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 445/886] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:02:15 2022] POST /identity/v3/projects => generated 361 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:02:17.450291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd1707-0202-4912-ae84-90bc45764a42 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:17.450868 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd1707-0202-4912-ae84-90bc45764a42 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:17.450868 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd1707-0202-4912-ae84-90bc45764a42 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:17.452202 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 442/887] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:17.458083 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:17.458523 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:17.458632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:17.477145 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:17.478331 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:17.538203 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:17.544155 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 446/888] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:17.810646 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd1707-0202-4912-ae84-90bc45764a42 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:17.907478 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd1707-0202-4912-ae84-90bc45764a42 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:17.940373 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a9a3-d5f9-4cf1-be82-4d0d1e446a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:17.940990 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a9a3-d5f9-4cf1-be82-4d0d1e446a9b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:17.941270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a9a3-d5f9-4cf1-be82-4d0d1e446a9b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:17.941541 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a9a3-d5f9-4cf1-be82-4d0d1e446a9b None admin] PATH_INFO: `/v3/domains/7dfe0ad7c4864d7e9a8a5280f4e1b193` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:17.942652 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a9a3-d5f9-4cf1-be82-4d0d1e446a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=7dfe0ad7c4864d7e9a8a5280f4e1b193)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:17.944090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a9a3-d5f9-4cf1-be82-4d0d1e446a9b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:17.945620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 443/889] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Oct 17 06:02:17 2022] GET /identity/v3/domains/7dfe0ad7c4864d7e9a8a5280f4e1b193 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:17.951202 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:18.047710 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4762e02-a5be-4215-9b75-c4ff17309b55 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:18.080314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13a355-f289-4902-ae5b-7ce8cb1ce21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:18.080856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a355-f289-4902-ae5b-7ce8cb1ce21f None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:18.081140 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a355-f289-4902-ae5b-7ce8cb1ce21f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:18.081414 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a355-f289-4902-ae5b-7ce8cb1ce21f None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:18.082074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13a355-f289-4902-ae5b-7ce8cb1ce21f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:18.083527 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13a355-f289-4902-ae5b-7ce8cb1ce21f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:18.212133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 447/890] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:02:17 2022] POST /identity/v3/users => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:02:20.355139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f03ec-fc95-4901-b20d-5c207bbdc19c None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:20.356281 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f03ec-fc95-4901-b20d-5c207bbdc19c None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:20.356281 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f03ec-fc95-4901-b20d-5c207bbdc19c None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:20.357610 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 444/891] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:20.362838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:20.363244 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:20.363420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:20.381184 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:20.382356 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:20.430124 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:20.447009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 448/892] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:20.712327 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f03ec-fc95-4901-b20d-5c207bbdc19c None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:20.773418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:20.811210 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f03ec-fc95-4901-b20d-5c207bbdc19c None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:20.843425 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:20.844042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:20.844319 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:20.844586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:20.854741 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:02:20.860040 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:02:20.861127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:02:20.862509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 445/893] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:02:20.866999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:20.962453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67c58a-d6a4-4d9a-ae78-a3841036b949 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:20.994997 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:20.995624 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:20.996491 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:20.996764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:20.997419 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:02:20.997717 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:20.999072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:21.006757 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 449/894] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:21.012253 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:21.108085 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab7c8f-e3fd-44a5-a2b9-91d0e4449b92 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:21.129497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:21.139602 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:21.140269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:21.140551 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:21.140820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] PATH_INFO: `/v3/users/6a98e769375d475ea9e9c43e3c38ae9c` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:21.153875 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] RBAC: Authorizing `identity:get_user(user_id=6a98e769375d475ea9e9c43e3c38ae9c)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:21.155280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:21.156845 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 446/895] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:02:21 2022] GET /identity/v3/users/6a98e769375d475ea9e9c43e3c38ae9c => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:21.162437 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:21.208054 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:21.259891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055dcf-fc1c-41fe-9b71-9b2459643a39 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:21.292217 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30117cd-869c-4b4a-ab60-3735a1de72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:21.293406 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30117cd-869c-4b4a-ab60-3735a1de72ad None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:21.293690 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30117cd-869c-4b4a-ab60-3735a1de72ad None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:21.293965 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30117cd-869c-4b4a-ab60-3735a1de72ad None admin] PATH_INFO: `/v3/projects/74776fed30844420b95ba01375abf367` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:21.295080 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30117cd-869c-4b4a-ab60-3735a1de72ad None admin] RBAC: Authorizing `identity:get_project(project_id=74776fed30844420b95ba01375abf367)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:21.296538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30117cd-869c-4b4a-ab60-3735a1de72ad None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:21.298143 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 450/896] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:02:21 2022] GET /identity/v3/projects/74776fed30844420b95ba01375abf367 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:21.303947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:21.399291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f197f37-b9ca-4a34-af03-cfb32d29460b None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:21.431605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7146a59-edac-42ce-aae4-d885d68e1065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:21.432497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146a59-edac-42ce-aae4-d885d68e1065 None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:21.432783 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146a59-edac-42ce-aae4-d885d68e1065 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:21.433055 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146a59-edac-42ce-aae4-d885d68e1065 None admin] PATH_INFO: `/v3/projects/74776fed30844420b95ba01375abf367/users/6a98e769375d475ea9e9c43e3c38ae9c/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:21.435567 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7146a59-edac-42ce-aae4-d885d68e1065 None admin] RBAC: Authorizing `identity:create_grant(project_id=74776fed30844420b95ba01375abf367, user_id=6a98e769375d475ea9e9c43e3c38ae9c, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:21.436965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7146a59-edac-42ce-aae4-d885d68e1065 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:21.509672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 447/897] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:02:21 2022] PUT /identity/v3/projects/74776fed30844420b95ba01375abf367/users/6a98e769375d475ea9e9c43e3c38ae9c/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:02:23.613194 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdded557-1587-4147-8e3b-5cfa9e9a00bd None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:23.613752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdded557-1587-4147-8e3b-5cfa9e9a00bd None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:23.613752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdded557-1587-4147-8e3b-5cfa9e9a00bd None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:23.615087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 451/898] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:02:23.620349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:23.620774 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:23.620901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:23.640041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:02:23.641227 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:02:23.675129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:23.705520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 448/899] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:02:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:02:23.975134 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdded557-1587-4147-8e3b-5cfa9e9a00bd None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:24.075424 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdded557-1587-4147-8e3b-5cfa9e9a00bd None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:24.088793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:24.108386 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:24.112829 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:24.112829 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:24.112829 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:24.119353 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:02:24.124707 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:02:24.125853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:02:24.127129 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 452/900] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:02:24.131739 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:24.226269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ade3f4-93a1-48ad-bbad-360fa1d3a032 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:24.259163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:24.259787 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:24.260066 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:24.260338 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:24.260993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:02:24.261292 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:24.263217 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:24.270877 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 449/901] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:24.276793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:24.373084 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0519f0-6a7d-445d-9f71-56a2eba65132 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:24.404827 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:24.405429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:24.405705 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:24.405973 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] PATH_INFO: `/v3/users/6a98e769375d475ea9e9c43e3c38ae9c` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:24.407074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] RBAC: Authorizing `identity:get_user(user_id=6a98e769375d475ea9e9c43e3c38ae9c)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:24.408514 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:24.410043 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 453/902] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Oct 17 06:02:24 2022] GET /identity/v3/users/6a98e769375d475ea9e9c43e3c38ae9c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:24.415672 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:24.450096 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:24.514603 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9e5b-8678-4d11-910c-81861ebed240 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:24.546597 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:24.547201 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:24.547504 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:24.547775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:24.611462 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:24.612874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:24.644041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:24.706924 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 450/903] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:02:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:02:24.712841 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:24.755431 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:24.811008 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cb24d-639d-4480-ace9-d81d48f0ada6 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:24.843532 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e6924-3814-4524-9fc9-1f76cd45df52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:24.844150 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e6924-3814-4524-9fc9-1f76cd45df52 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:24.844432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e6924-3814-4524-9fc9-1f76cd45df52 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:24.844704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e6924-3814-4524-9fc9-1f76cd45df52 None admin] PATH_INFO: `/v3/projects/74776fed30844420b95ba01375abf367` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:24.845808 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e6924-3814-4524-9fc9-1f76cd45df52 None admin] RBAC: Authorizing `identity:get_project(project_id=74776fed30844420b95ba01375abf367)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:24.847220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e6924-3814-4524-9fc9-1f76cd45df52 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:24.848843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 454/904] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Oct 17 06:02:24 2022] GET /identity/v3/projects/74776fed30844420b95ba01375abf367 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:24.854797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:02:24.950251 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f26f-500e-4513-8ca0-6a21fa004064 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:02:24.983159 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:02:24.983779 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:02:24.984060 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:02:24.984331 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:02:24.985044 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=74776fed30844420b95ba01375abf367, user.id=6a98e769375d475ea9e9c43e3c38ae9c) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:02:24.985349 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:02:24.986756 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcc5c7-6d0d-4e92-a7ec-a53eab8d0332 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:02:24.988503 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:24.992802 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 451/905] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:02:24 2022] GET /identity/v3/role_assignments?user.id=6a98e769375d475ea9e9c43e3c38ae9c&scope.project.id=74776fed30844420b95ba01375abf367&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:02:34.838884 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=45 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:02:35.024079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:36.981089 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e063c-76d0-4ae2-b5f1-b7a88a452f14 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:36.981704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e063c-76d0-4ae2-b5f1-b7a88a452f14 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:36.981704 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e063c-76d0-4ae2-b5f1-b7a88a452f14 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:36.983103 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 455/906] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:03:36.988593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:36.989041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:36.989254 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:37.007965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:03:37.012086 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:03:37.076209 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 452/907] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:03:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:03:37.349522 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e063c-76d0-4ae2-b5f1-b7a88a452f14 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:37.449721 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e063c-76d0-4ae2-b5f1-b7a88a452f14 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:37.486156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:37.486787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:37.487072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:37.487372 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:37.498100 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:37.500590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:37.513838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:37.515127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:37.516264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:37.516264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:37.516264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:37.516264 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 456/908] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:37.521140 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:37.618965 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f6a35-d6e8-4755-ae96-4fed98233103 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:37.646631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:37.652971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:37.653577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:37.653856 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:37.654127 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:37.654764 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:37.655065 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:37.656451 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383cb1e-2564-4f42-8271-936c2ee4ee2c None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:37.667288 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 453/909] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:37.673006 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:37.726788 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:37.775488 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dab851-7647-4159-9031-90167092bb84 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:37.809450 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a5c7e-9c0f-477d-b985-b635827076b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:37.810053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a5c7e-9c0f-477d-b985-b635827076b1 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:37.810333 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a5c7e-9c0f-477d-b985-b635827076b1 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:37.810609 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a5c7e-9c0f-477d-b985-b635827076b1 None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:37.811321 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a5c7e-9c0f-477d-b985-b635827076b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:37.812745 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a5c7e-9c0f-477d-b985-b635827076b1 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:37.897573 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 457/910] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Oct 17 06:03:37 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:03:40.049087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad3138-431e-49da-92c5-f2a063b9032f None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:40.059493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad3138-431e-49da-92c5-f2a063b9032f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:40.059493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad3138-431e-49da-92c5-f2a063b9032f None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:40.059493 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 454/911] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:03:40.059818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:40.059818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:40.059818 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:40.077065 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:03:40.078302 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:03:40.144296 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 458/912] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:03:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:03:40.410837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad3138-431e-49da-92c5-f2a063b9032f None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:40.512133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad3138-431e-49da-92c5-f2a063b9032f None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:40.545719 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:40.546381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:40.546749 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:40.547114 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:40.557804 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:03:40.562872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:40.564130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:40.565397 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 455/913] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:03:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:40.570149 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:40.609339 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:40.666503 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf15719-ea80-46ae-be2e-c164631fbf63 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:40.700292 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:40.700893 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:40.701168 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:40.701435 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:40.702094 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] RBAC: Adding query filter params (name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:40.702393 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:40.703802 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:40.711401 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 459/914] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:03:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:40.717795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:40.766855 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:40.815540 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75c01d-3476-4ed0-ac4c-b5fe900613fd None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:40.848325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:40.848944 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:40.849211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:40.849486 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:40.860178 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:40.861601 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:40.875957 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:40.877139 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:40.878250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:40.878250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:40.878250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:40.878250 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 456/915] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:40.883015 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:40.977123 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:40.980673 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0ab76-dcc9-4649-a902-ba55ef209069 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.013837 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.014445 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.014724 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.015000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.015674 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:41.015977 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.017328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.028076 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 460/916] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:41.033931 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:41.131602 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f221abb-10cd-4d8a-a60c-6bf65bdf81c2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.133108 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:41.164166 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.165359 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.165647 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.165931 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] PATH_INFO: `/v3/users/placement` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.173337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.174755 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:03:41.186724 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:41.187910 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:41.188992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 457/917] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:41.193860 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:41.294320 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adcb96-b435-4f87-bb9a-a0989e3c37d9 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.327867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.328466 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.328750 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.329028 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.329757 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:41.330142 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.331603 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.349051 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 461/918] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 06:03:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:41.355245 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:41.454487 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96444-4f9e-489d-94a4-134fb7677412 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.488578 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.489200 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.489475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.489745 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.500121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.501581 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:41.515337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:41.519023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:41.520119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:41.520119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:41.520119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:41.520119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 458/919] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:41.522641 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:41.542271 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:41.623262 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e85cc-efb5-4847-bfa3-9f89f99b173b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.657769 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.658382 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.658667 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.658940 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.659625 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:41.659930 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.661298 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.673323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 462/920] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:41.679316 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:41.695659 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:41.778088 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc546d-0432-4686-bac7-e77d4600ffcc None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.812371 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.813028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.813314 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.813586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.820320 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.821806 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:03:41.833198 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:41.834226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:41.835375 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 459/921] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:41.843571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:41.919752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:41.940918 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cc2ef-adad-49bf-9ec0-89d6d3cb655f None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:41.973405 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:41.974009 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:41.974291 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:41.974565 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:41.975258 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:41.975590 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:41.977014 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0e468-b95e-4f7f-8c02-237662cad63a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:41.989309 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 463/922] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:41.995627 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:42.071558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:42.095194 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1db69-5060-49cc-8511-c6217580a824 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:42.127524 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a06f5a-1b8b-42a6-9bcc-2f23f31db0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:42.128167 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a06f5a-1b8b-42a6-9bcc-2f23f31db0df None admin] REQUEST_METHOD: `PUT` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:42.128445 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a06f5a-1b8b-42a6-9bcc-2f23f31db0df None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:42.128715 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a06f5a-1b8b-42a6-9bcc-2f23f31db0df None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/e588341177544d43a4bdf82cc1107db1/roles/2e80dbe249184534b7014f12334c7c48` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:42.142544 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a06f5a-1b8b-42a6-9bcc-2f23f31db0df None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=e588341177544d43a4bdf82cc1107db1, role_id=2e80dbe249184534b7014f12334c7c48)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:42.144000 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a06f5a-1b8b-42a6-9bcc-2f23f31db0df None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:42.262345 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 460/923] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:03:41 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/e588341177544d43a4bdf82cc1107db1/roles/2e80dbe249184534b7014f12334c7c48 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:03:44.388785 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30b88-8a7d-4668-8398-e50969fdf894 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:44.389403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30b88-8a7d-4668-8398-e50969fdf894 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:44.389403 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30b88-8a7d-4668-8398-e50969fdf894 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:44.390649 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 464/924] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:03:44.396130 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:44.396520 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:44.396668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:44.415704 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:03:44.416878 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:03:44.482689 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 461/925] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:03:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:03:44.748371 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30b88-8a7d-4668-8398-e50969fdf894 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:44.851713 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30b88-8a7d-4668-8398-e50969fdf894 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:44.886814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885b231-2394-4756-9692-731082afdad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:44.887485 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885b231-2394-4756-9692-731082afdad4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:44.887769 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885b231-2394-4756-9692-731082afdad4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:44.888047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885b231-2394-4756-9692-731082afdad4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:44.894941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:44.899062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885b231-2394-4756-9692-731082afdad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-8885b231-2394-4756-9692-731082afdad4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:03:44.904042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:44.908795 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:44.910784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 465/926] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Oct 17 06:03:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:44.912133 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:44.966079 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:45.009195 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fa602-aabf-45c1-9c82-1116eada1415 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:45.043877 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:45.044485 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:45.044758 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:45.045028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:45.045679 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] RBAC: Adding query filter params (name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:45.045978 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:45.047332 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:45.055063 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 462/927] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Oct 17 06:03:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:45.061768 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885b231-2394-4756-9692-731082afdad4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:45.159912 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885b231-2394-4756-9692-731082afdad4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:45.193544 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:45.194165 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:45.194477 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:45.194750 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:45.204548 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:45.205971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:45.220934 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:45.222409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:45.223653 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:45.223653 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:45.223653 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:45.223653 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 466/928] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:45.228405 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:45.327996 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c1f81-6d08-4c55-9fa6-16411ecf0921 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:45.344211 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:45.361947 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:45.362558 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:45.362837 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:45.363109 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:45.363928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:45.364232 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:45.365593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:45.378091 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 463/929] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:45.384615 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:45.436104 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:45.487493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f21b70-38ed-4c2f-a897-0092ddf01998 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:45.521081 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:45.521705 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:45.521988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:45.522257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:45.529316 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:45.530741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:03:45.542731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:45.543835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:45.544906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 467/930] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:45.549770 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:45.646700 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347bbbc-485d-4f71-8959-308104276f33 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:45.679639 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:45.680797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:45.681075 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:45.681343 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:45.682058 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:45.682357 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:45.684087 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:45.701062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 464/931] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 06:03:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:45.706814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:45.806029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf32d8-5c52-4447-a1f8-917bc7bcd3eb None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:45.816534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:45.839126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:45.840563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:45.840563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:45.840563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:45.906842 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:45.908234 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:46.007903 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 468/932] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:03:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:03:46.014381 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:46.047091 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:46.110971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d52395-4187-4267-b8ae-c1e37e1c3e6d None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:46.143410 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:46.144028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:46.144305 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:46.144574 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:46.154329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:46.155778 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:46.170616 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:46.171842 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:46.172847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:46.172847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:46.172847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:46.172847 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 465/933] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:03:46.177359 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:46.189507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:46.274409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb4178-fa7b-4105-82bb-3a5940b5c7b4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:46.308693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:46.309293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:46.309571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:46.309845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:46.310480 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:46.310777 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:46.312156 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:46.323159 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 469/934] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:03:46.328795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:46.411216 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:46.426225 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc898b3-06c8-4b89-bbbc-08b9016f88d2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:46.460684 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:46.461343 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:46.461626 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:46.461897 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:46.470605 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:46.472064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:03:46.483671 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:46.484658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:46.485623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 466/935] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:03:46.490627 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:46.559193 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:46.589170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f0175-93a5-423c-901c-dd9b8af202f2 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:46.622002 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:46.624847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:46.624847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:46.624847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] PATH_INFO: `/v3/projects` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:46.624847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:46.624847 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:46.625556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e5a1-b45e-4823-a2d0-615d4de774a5 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:46.638032 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 470/936] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:46.644186 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:46.744317 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478d4e-4fb0-47af-82d4-34a8d4e980ff None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:46.778310 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:46.778926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:46.779207 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:46.779509 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:46.780237 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] RBAC: Adding query filter params (role.id=2e80dbe249184534b7014f12334c7c48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=e588341177544d43a4bdf82cc1107db1) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:46.780547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:46.782093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbac55-1073-4134-9025-2f053eed68f5 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:46.787887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 467/937] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:03:46 2022] GET /identity/v3/role_assignments?user.id=e588341177544d43a4bdf82cc1107db1&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=2e80dbe249184534b7014f12334c7c48 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:49.063456 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9aea-caf5-44d5-8e82-63e52c5a0bfc None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:49.064000 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9aea-caf5-44d5-8e82-63e52c5a0bfc None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:49.064131 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9aea-caf5-44d5-8e82-63e52c5a0bfc None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:49.065388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 471/938] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:03:49.071516 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:49.071969 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:49.072071 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:49.090340 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:03:49.091559 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:03:49.094331 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:49.157663 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 468/939] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:03:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:03:49.431846 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9aea-caf5-44d5-8e82-63e52c5a0bfc None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:49.541843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9aea-caf5-44d5-8e82-63e52c5a0bfc None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:49.572811 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:49.578762 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:49.579560 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:49.579843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:49.580121 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:49.591804 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:03:49.596693 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:49.597775 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:49.598732 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 472/940] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:03:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:49.605552 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:49.707169 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a94aa-c39a-4bab-aeb8-641ead9040e9 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:49.740955 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:49.741586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:49.741869 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:49.742162 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] PATH_INFO: `/v3/roles` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:49.743364 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] RBAC: Adding query filter params (name=admin) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:49.743674 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:49.745037 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:49.753284 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 469/941] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:49.759774 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:49.859624 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448049e6-33e8-4875-9c68-d941e29cb457 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:49.894723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:49.895388 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:49.895671 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:49.896006 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:49.909101 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:49.910556 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:49.925859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:49.927047 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:49.928085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:49.928085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:49.928085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:49.928085 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 473/942] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:49.934028 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:49.937082 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:50.031872 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720d3eb-93b0-48ea-83ba-138ad45de114 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:50.065721 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:50.066332 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:50.066614 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:50.066887 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:50.067575 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:50.067882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:50.069307 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:50.082047 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 470/943] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:50.088087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:50.133638 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:50.191263 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93414a-5029-46a3-92e0-b31a4b741b23 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:50.224542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:50.225171 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:50.225448 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:50.225716 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:50.234334 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:50.235797 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:03:50.247501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:50.248701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:50.250152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 474/944] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:50.254514 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:50.319841 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:50.357771 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966da88-28bd-4695-a6d7-78afe39bb8a2 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:50.393504 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:50.394107 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:50.394384 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:50.394657 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] PATH_INFO: `/v3/users` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:50.395413 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:50.395718 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:50.397128 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:50.413995 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 471/945] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 06:03:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:50.420070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:50.532801 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c0dbf-3138-4ed2-a534-750f32f14b1a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:50.534453 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:50.570633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:50.574525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:50.574525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:50.574525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:50.582588 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:50.584071 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:50.599979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:50.601259 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:50.602370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:50.602370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:50.602370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:50.602370 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 475/946] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:03:50.607113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:50.712158 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97246e2d-5c4f-432e-816f-556de9a3ecc8 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:50.731901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:50.745654 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:50.746274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:50.746554 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:50.746825 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:50.747507 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:50.747813 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:50.749187 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:50.761638 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 472/947] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:50.767857 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:50.872946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e349cca-0592-4e32-9264-959e7a43ad5e None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:50.906276 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:50.907018 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:50.907326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:50.907593 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:50.914797 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:50.916243 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:03:50.927409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:50.928501 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:50.929493 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 476/948] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:50.935633 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:51.044608 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a20ba-d79b-4edf-b0a6-457466faca50 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:51.077861 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:51.078471 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:51.078788 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:51.079058 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:51.079775 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:51.080078 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:51.081491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809c661-7acb-4358-84b2-1ce1ee3e503a None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:51.093960 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 473/949] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:03:51.100150 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:51.116901 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:51.204251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaaf9a-a700-424e-aef4-b6c65affdcb4 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:51.239182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541bd06-2d65-45cb-9c81-b1e4b14504fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:51.239875 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541bd06-2d65-45cb-9c81-b1e4b14504fc None admin] REQUEST_METHOD: `PUT` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:51.240157 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541bd06-2d65-45cb-9c81-b1e4b14504fc None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:51.240429 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541bd06-2d65-45cb-9c81-b1e4b14504fc None admin] PATH_INFO: `/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/e588341177544d43a4bdf82cc1107db1/roles/7b1a25a6880546179dbb64fdb8a63e48` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:51.242899 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541bd06-2d65-45cb-9c81-b1e4b14504fc None admin] RBAC: Authorizing `identity:create_grant(project_id=6727b897172440d8bdc7c2d81ea3d9da, user_id=e588341177544d43a4bdf82cc1107db1, role_id=7b1a25a6880546179dbb64fdb8a63e48)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:51.244392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541bd06-2d65-45cb-9c81-b1e4b14504fc None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:51.332282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 477/950] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Oct 17 06:03:51 2022] PUT /identity/v3/projects/6727b897172440d8bdc7c2d81ea3d9da/users/e588341177544d43a4bdf82cc1107db1/roles/7b1a25a6880546179dbb64fdb8a63e48 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 06:03:53.489189 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8a2d1-1135-4a8b-b33a-eac7e19acb0b None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:53.489717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8a2d1-1135-4a8b-b33a-eac7e19acb0b None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:53.489717 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8a2d1-1135-4a8b-b33a-eac7e19acb0b None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:53.491062 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 474/951] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:03:53.496566 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:53.497003 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:53.497263 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:53.515803 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:03:53.517015 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:03:53.586749 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 478/952] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:03:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:03:53.862992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8a2d1-1135-4a8b-b33a-eac7e19acb0b None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:53.905998 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:53.964448 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8a2d1-1135-4a8b-b33a-eac7e19acb0b None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.001403 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.002050 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.002328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.002600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:54.015805 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 06:03:54.020720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:54.021928 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:54.023094 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 475/953] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Oct 17 06:03:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 06:03:54.031126 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:54.042633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:54.131274 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611084b-4783-42fb-9f9b-fd4fed92e912 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.166267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.167218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.168366 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.168699 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] PATH_INFO: `/v3/roles` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:54.169434 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] RBAC: Adding query filter params (name=admin) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:54.169799 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:54.171257 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:54.180136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 479/954] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Oct 17 06:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:54.186405 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:54.282440 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:54.285742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89759c3-3229-494a-bd32-d90043941349 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.319393 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.320042 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.320323 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.320627 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:54.331110 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:54.332593 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:54.349859 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:54.352732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:54.353711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:54.353711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:54.353711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:54.353711 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 476/955] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:54.357686 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:54.423888 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:54.458530 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8e070-6abd-4874-91f1-94362982823b None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.494531 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.495136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.495444 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.495763 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] PATH_INFO: `/v3/domains` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:54.496404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:54.496703 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:54.498064 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:54.511985 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 480/956] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:54.518064 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:54.619413 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9692506-0e51-4206-a56b-2eada95a12dd None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.651797 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.652475 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.652753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.653021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:54.660378 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:54.661807 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 06:03:54.673518 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:54.681172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:54.682453 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 477/957] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:54.683558 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:54.792268 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ead7a8-67d6-4eee-b529-2abcb43d3480 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.818409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:54.827404 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.828015 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.828297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.828572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] PATH_INFO: `/v3/users` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:54.829300 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:54.829605 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:54.831026 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:54.848314 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 481/958] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Oct 17 06:03:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:54.855000 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:54.870271 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:54.954527 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0037c-1a69-4a08-8413-9c3e600c38a7 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:54.988736 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:54.989350 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:54.989632 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:54.989906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:55.056705 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:55.058422 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:55.071584 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:55.164542 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 478/959] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Oct 17 06:03:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 06:03:55.170507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:55.274361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bf50f-0434-41a7-af01-43a9efad694d None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:55.310053 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:55.310689 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:55.310974 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:55.311251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:55.321719 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:55.323173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 06:03:55.337938 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:55.339164 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 06:03:55.340224 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 06:03:55.340224 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 06:03:55.340224 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:55.340224 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 482/960] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Oct 17 06:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:55.345112 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:55.444827 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e1619-4028-429c-8718-51d5b70cb875 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:55.458046 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:55.479951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:55.484132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:55.484132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:55.484132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:55.484132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:55.484132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:55.484132 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:55.500293 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 479/961] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Oct 17 06:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:55.501211 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:55.540760 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:55.605742 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e869dbe-b573-4bf5-9d81-5db4d442d35a None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:55.644345 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:55.644992 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:55.645500 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:55.645787 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:55.652978 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:55.654420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 06:03:55.666793 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:55.667975 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: ERROR keystone.server.flask.application Oct 17 06:03:55.668968 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 483/962] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Oct 17 06:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:55.674113 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:55.777297 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d798c-85f5-4d24-81d0-ac8f6684c5a3 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:55.811191 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:55.811728 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:55.812007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:55.812280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] PATH_INFO: `/v3/projects` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:55.812971 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:55.813275 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:55.814701 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f6884-91e0-4f57-9e97-3502fbdec841 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:55.827176 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 480/963] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Oct 17 06:03:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:55.833438 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:55.925741 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:55.932013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d03f40-ce33-4fc2-83a8-1260963a3cf8 None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:55.965551 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:55.966462 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:55.967010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:55.967208 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:55.968034 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] RBAC: Adding query filter params (role.id=7b1a25a6880546179dbb64fdb8a63e48, scope.project.id=6727b897172440d8bdc7c2d81ea3d9da, user.id=e588341177544d43a4bdf82cc1107db1) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:55.968729 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:55.970167 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491c6f-dcfc-4964-8a69-5e870eb80332 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:55.976029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 484/964] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Oct 17 06:03:55 2022] GET /identity/v3/role_assignments?user.id=e588341177544d43a4bdf82cc1107db1&scope.project.id=6727b897172440d8bdc7c2d81ea3d9da&role.id=7b1a25a6880546179dbb64fdb8a63e48 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 06:03:58.146769 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a0cf-2c6d-42dc-8b70-73d2bcc10b9a None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:58.147435 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a0cf-2c6d-42dc-8b70-73d2bcc10b9a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:58.147435 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a0cf-2c6d-42dc-8b70-73d2bcc10b9a None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:58.148753 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 481/965] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:03:58.154269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:58.154688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:58.154803 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:58.173723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:03:58.174922 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:03:58.241230 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 485/966] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:03:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:03:58.512485 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703a0cf-2c6d-42dc-8b70-73d2bcc10b9a None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:58.545517 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:58.614798 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703a0cf-2c6d-42dc-8b70-73d2bcc10b9a None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:58.650953 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:58.651634 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:58.651909 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:58.652174 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] PATH_INFO: `/v3/services/placement` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:58.652761 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:58.654131 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:03:58.663921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:03:58.667539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:03:58.668612 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 482/967] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:03:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:03:58.671871 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:58.763062 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:58.772845 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814923d-0a36-4f04-ad33-6ae9f93a5287 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:58.807606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:58.808474 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:58.808756 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:58.809030 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:58.809668 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] RBAC: Adding query filter params (name=placement) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:58.809971 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:58.811420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0f6f-9bea-40d9-a7f5-31ef9147267f None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:58.821239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 486/968] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 06:03:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:03:58.822292 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:03:58.919560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:03:58.925421 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c832bf-f55c-4f3b-b0c8-eb585ee42c1f None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:03:58.959415 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:03:58.960022 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:03:58.960297 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:03:58.960564 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] PATH_INFO: `/v3/services` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:03:58.961825 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] RBAC: Adding query filter params (type=placement) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:03:58.962121 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:03:58.963501 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358792f-3e43-4ad9-9449-963ceada3c62 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:03:58.967895 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 483/969] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 06:03:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:04:01.161274 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b670f5-9a26-47b7-95f7-4193f0d2b68e None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:01.161838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b670f5-9a26-47b7-95f7-4193f0d2b68e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:01.161838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b670f5-9a26-47b7-95f7-4193f0d2b68e None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:01.163343 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 487/970] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:04:01.169692 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2193290-b64e-4137-8e6c-f9e70d44e8b3 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:01.170224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2193290-b64e-4137-8e6c-f9e70d44e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:01.170224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2193290-b64e-4137-8e6c-f9e70d44e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:01.189485 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-a2193290-b64e-4137-8e6c-f9e70d44e8b3 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:04:01.190702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-a2193290-b64e-4137-8e6c-f9e70d44e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:04:01.259803 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 484/971] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:04:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:04:01.531238 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b670f5-9a26-47b7-95f7-4193f0d2b68e None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:01.635892 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b670f5-9a26-47b7-95f7-4193f0d2b68e None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:01.671392 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274376-9f90-4550-aef6-40f5bcebfc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:01.672001 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274376-9f90-4550-aef6-40f5bcebfc94 None admin] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:01.672284 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274376-9f90-4550-aef6-40f5bcebfc94 None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:01.672557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274376-9f90-4550-aef6-40f5bcebfc94 None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:01.673225 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274376-9f90-4550-aef6-40f5bcebfc94 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:01.674591 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274376-9f90-4550-aef6-40f5bcebfc94 None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:01.772372 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 488/972] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Oct 17 06:04:01 2022] POST /identity/v3/services => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:04:03.948785 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ae677-16f4-4eee-be39-e28fd504822a None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:03.949351 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ae677-16f4-4eee-be39-e28fd504822a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:03.949351 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ae677-16f4-4eee-be39-e28fd504822a None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:03.950676 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 485/973] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:04:03.956770 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:03.957323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:03.957323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:03.976356 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:04:03.977572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:04:03.987570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:04.052362 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 489/974] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:04:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:04:04.326097 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ae677-16f4-4eee-be39-e28fd504822a None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:04.418090 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:04.428513 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ae677-16f4-4eee-be39-e28fd504822a None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:04.464772 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:04.465425 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:04.465732 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:04.466005 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] PATH_INFO: `/v3/services/placement` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:04.466623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:04.468129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:04:04.477287 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:04:04.478586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:04:04.484136 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 486/975] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:04:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:04:04.485049 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:04.584206 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f66918-960c-4402-ad9f-48fa0cc80df7 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:04.619933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:04.620606 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:04.620890 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:04.621163 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:04.621853 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] RBAC: Adding query filter params (name=placement) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:04:04.622218 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:04.623676 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:04.629267 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 490/976] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 06:04:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:04:04.635692 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:04.735746 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818da3f-72db-4c05-ba9e-b4ea64d02c03 None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:04.770225 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:04.770917 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:04.771281 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:04.771656 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:04.772355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] RBAC: Adding query filter params (interface=public, service_id=42f59889e4ac42b297fa3327b2a26f72, region_id=RegionOne) {{(pid=218502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:04:04.772683 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:04.774106 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93b6e2-cdc3-4824-ac07-6a86e8f5dbf9 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:04.779204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 487/977] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Oct 17 06:04:04 2022] GET /identity/v3/endpoints?service_id=42f59889e4ac42b297fa3327b2a26f72&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:04:04.785104 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:04.789950 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:04.885633 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4d4b-f62d-4756-8d8b-828cd8a607db None admin] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:04.920066 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad543d89-e5e1-4be0-9157-7d7fe887ac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:04.920682 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad543d89-e5e1-4be0-9157-7d7fe887ac8c None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:04.920966 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad543d89-e5e1-4be0-9157-7d7fe887ac8c None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:04.921242 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad543d89-e5e1-4be0-9157-7d7fe887ac8c None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:04.921859 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad543d89-e5e1-4be0-9157-7d7fe887ac8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:04.923239 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad543d89-e5e1-4be0-9157-7d7fe887ac8c None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:04.930608 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 491/978] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Oct 17 06:04:04 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 06:04:07.121851 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8a168-66b5-40fa-abfd-654c827cfd8d None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:07.122442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8a168-66b5-40fa-abfd-654c827cfd8d None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:07.122442 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8a168-66b5-40fa-abfd-654c827cfd8d None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:07.123902 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 488/979] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:04:07.129389 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:07.129856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:07.129970 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:07.148057 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:04:07.149409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:04:07.214335 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:07.219933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 492/980] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:04:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 06:04:07.497893 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8a168-66b5-40fa-abfd-654c827cfd8d None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:07.517633 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:07.602906 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8a168-66b5-40fa-abfd-654c827cfd8d None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:07.636243 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:07.636878 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:07.637157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:07.637430 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] PATH_INFO: `/v3/services/placement` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:07.638015 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:07.639400 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 06:04:07.648713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 06:04:07.649882 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application with Lock( Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self._enter() Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application return self.creator() Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application created_value = creator( Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: ERROR keystone.server.flask.application Oct 17 06:04:07.651291 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 489/981] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Oct 17 06:04:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 06:04:07.655941 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:07.754844 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71673e-7cec-407c-b32d-04e07cd127c2 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:07.789946 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:07.790555 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:07.790835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:07.791107 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] PATH_INFO: `/v3/services` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:07.791773 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] RBAC: Adding query filter params (name=placement) {{(pid=218501) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 06:04:07.792072 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:07.793436 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474c85d-95e7-41ef-8ae8-731fa6d4763e None admin] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:07.798511 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 493/982] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Oct 17 06:04:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 06:04:07.805032 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:04:07.887677 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:07.904668 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe82e51-786f-4107-9ef4-e3f93a77f46e None admin] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:04:07.938680 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e3bc-93fc-406d-85cb-cb96b30b7a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf714812a1a846729a9dd3749da0e163', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:04:07.939288 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e3bc-93fc-406d-85cb-cb96b30b7a88 None admin] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:04:07.939620 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e3bc-93fc-406d-85cb-cb96b30b7a88 None admin] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:04:07.939892 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e3bc-93fc-406d-85cb-cb96b30b7a88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:04:07.940560 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e3bc-93fc-406d-85cb-cb96b30b7a88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:04:07.941922 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e3bc-93fc-406d-85cb-cb96b30b7a88 None admin] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:04:08.075681 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 490/983] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Oct 17 06:04:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 06:04:17.795560 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=69 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:04:18.070751 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:14.565706 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57732d65-2324-4183-ae8d-b517ea2fc50b None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:14.566926 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57732d65-2324-4183-ae8d-b517ea2fc50b None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:14.567487 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57732d65-2324-4183-ae8d-b517ea2fc50b None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:14.575738 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0dae9-cc88-4975-9af6-fe114fe54aec None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:14.576640 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0dae9-cc88-4975-9af6-fe114fe54aec None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:14.577569 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0dae9-cc88-4975-9af6-fe114fe54aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:14.580628 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 494/984] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:14.684779 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-a4f0dae9-cc88-4975-9af6-fe114fe54aec None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:14.694352 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-a4f0dae9-cc88-4975-9af6-fe114fe54aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:14.935455 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 491/985] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:15.298282 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b76d99-d161-4b2c-8e40-6b1fd09fd7a4 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:15.298937 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b76d99-d161-4b2c-8e40-6b1fd09fd7a4 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:15.299642 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b76d99-d161-4b2c-8e40-6b1fd09fd7a4 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:15.301571 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 495/986] 10.222.0.13 () {54 vars in 964 bytes} [Mon Oct 17 06:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:15.308214 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f404c05-aaf1-44b0-90de-4acada77a90c None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:15.308499 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f404c05-aaf1-44b0-90de-4acada77a90c None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:15.308773 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f404c05-aaf1-44b0-90de-4acada77a90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:15.564049 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-5f404c05-aaf1-44b0-90de-4acada77a90c None None] MFA Rules not processed for user `0a76012127184a19bd9ccab445d24d75`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:15.565286 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-5f404c05-aaf1-44b0-90de-4acada77a90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:15.616687 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:15.791033 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 492/987] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Oct 17 06:13:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:15.792136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b76d99-d161-4b2c-8e40-6b1fd09fd7a4 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:15.794494 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-d7b76d99-d161-4b2c-8e40-6b1fd09fd7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:16.125882 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b76d99-d161-4b2c-8e40-6b1fd09fd7a4 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:16.304439 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:16.305070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:16.305327 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:16.305688 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:16.307253 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:16.526982 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:16.653373 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:16.655024 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97015379-bfc9-4e73-8854-294a86141d47 service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:16.711943 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 496/988] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 921 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 06:13:19.574017 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca4f2-5a56-46ca-8042-b74b03cae5b2 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:19.574563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca4f2-5a56-46ca-8042-b74b03cae5b2 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:19.574563 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca4f2-5a56-46ca-8042-b74b03cae5b2 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:19.581297 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d753d-31fd-4786-8b82-a2b6ca66c7bb None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:19.581764 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d753d-31fd-4786-8b82-a2b6ca66c7bb None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:19.581856 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d753d-31fd-4786-8b82-a2b6ca66c7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:19.586433 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 493/989] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:19.669677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-509d753d-31fd-4786-8b82-a2b6ca66c7bb None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:19.671321 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-509d753d-31fd-4786-8b82-a2b6ca66c7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:19.785143 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 497/990] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:13:20.072100 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca4f2-5a56-46ca-8042-b74b03cae5b2 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:20.073632 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-dadca4f2-5a56-46ca-8042-b74b03cae5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:20.214153 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca4f2-5a56-46ca-8042-b74b03cae5b2 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:20.296543 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:20.298852 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:20.299513 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:20.299799 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:20.300073 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:20.301795 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:20.427131 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:20.428610 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0460ef-ca5e-43f9-b72a-7b16663aac8e service neutron] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:20.449550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 494/991] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:23.199709 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec63a5-f145-41d0-bdaa-3d76615f9e67 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:23.200251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec63a5-f145-41d0-bdaa-3d76615f9e67 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:23.200251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec63a5-f145-41d0-bdaa-3d76615f9e67 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:23.201613 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 498/992] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:23.206981 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400bd40-67ca-4488-a475-e49dd1adff48 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:23.207432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400bd40-67ca-4488-a475-e49dd1adff48 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:23.207568 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400bd40-67ca-4488-a475-e49dd1adff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:23.273803 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-c400bd40-67ca-4488-a475-e49dd1adff48 None None] MFA Rules not processed for user `c4cc297532144f6f8f4534b8faf80b42`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:23.275528 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-c400bd40-67ca-4488-a475-e49dd1adff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:23.330007 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:23.376367 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 495/993] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:23.661727 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec63a5-f145-41d0-bdaa-3d76615f9e67 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:23.663270 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-2bec63a5-f145-41d0-bdaa-3d76615f9e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:23.674488 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:23.771658 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec63a5-f145-41d0-bdaa-3d76615f9e67 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:23.848389 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:23.849013 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:23.849293 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:23.849563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:23.851182 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:23.915241 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:23.962132 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:23.963592 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deff239-5dd3-4c1d-9f06-0b4bb6584e41 service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:23.981955 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 499/994] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:27.028702 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990731b-3dbd-4979-9330-4ad29a840ba1 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:27.029274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990731b-3dbd-4979-9330-4ad29a840ba1 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:27.029274 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990731b-3dbd-4979-9330-4ad29a840ba1 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:27.030592 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 496/995] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:27.035951 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7409850-c26e-4c7d-9b5e-72d7e1f3c06f None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:27.036398 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7409850-c26e-4c7d-9b5e-72d7e1f3c06f None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:27.036498 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7409850-c26e-4c7d-9b5e-72d7e1f3c06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:27.101170 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-c7409850-c26e-4c7d-9b5e-72d7e1f3c06f None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:27.102908 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-c7409850-c26e-4c7d-9b5e-72d7e1f3c06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:27.207585 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 500/996] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:27.494328 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990731b-3dbd-4979-9330-4ad29a840ba1 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:27.495976 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-d990731b-3dbd-4979-9330-4ad29a840ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:27.601959 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990731b-3dbd-4979-9330-4ad29a840ba1 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:27.676399 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:27.677016 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:27.677293 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:27.677562 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:27.679168 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:27.688129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:27.794631 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:27.796021 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817dd4-8972-4783-ab6a-81c4d52e92ea service neutron] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:27.814534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 497/997] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:31.338328 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f7af-15a5-4786-9049-4e2f0d057ba2 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:31.338909 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f7af-15a5-4786-9049-4e2f0d057ba2 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:31.338909 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f7af-15a5-4786-9049-4e2f0d057ba2 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:31.340553 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 501/998] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:31.345572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6e87-d4bb-4c9f-be3f-e0aa08787906 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:31.345973 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6e87-d4bb-4c9f-be3f-e0aa08787906 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:31.348038 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6e87-d4bb-4c9f-be3f-e0aa08787906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:31.420688 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-827e6e87-d4bb-4c9f-be3f-e0aa08787906 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:31.422233 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-827e6e87-d4bb-4c9f-be3f-e0aa08787906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:31.489348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:31.535408 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 498/999] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:31.819354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f7af-15a5-4786-9049-4e2f0d057ba2 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:31.821547 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-5224f7af-15a5-4786-9049-4e2f0d057ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:31.847854 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:31.938973 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f7af-15a5-4786-9049-4e2f0d057ba2 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:32.022721 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:32.023476 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:32.023864 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:32.024220 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:32.025891 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:32.104557 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:32.143498 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:32.145036 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8edd75-6262-46d1-a584-811ab413cfcd service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:32.165127 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 502/1000] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:34.423270 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017c01-2d13-4ea0-b0d5-cb440a91b2fd None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:34.423898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017c01-2d13-4ea0-b0d5-cb440a91b2fd None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:34.423898 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017c01-2d13-4ea0-b0d5-cb440a91b2fd None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:34.425335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 499/1001] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:34.431251 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0066eb-3261-4028-9cd7-0e778ebadefc None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:34.431698 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0066eb-3261-4028-9cd7-0e778ebadefc None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:34.431875 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0066eb-3261-4028-9cd7-0e778ebadefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:34.508236 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-ab0066eb-3261-4028-9cd7-0e778ebadefc None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:34.509462 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-ab0066eb-3261-4028-9cd7-0e778ebadefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:34.618152 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 503/1002] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 06:13:34.919810 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c5f26-7308-4c4c-bf8f-40484b10f6d8 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:34.920093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c5f26-7308-4c4c-bf8f-40484b10f6d8 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:34.920363 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c5f26-7308-4c4c-bf8f-40484b10f6d8 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:34.921706 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 500/1003] 10.222.0.13 () {54 vars in 964 bytes} [Mon Oct 17 06:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:34.927874 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84203-a30d-4b58-95a6-cb0d40e38ccd None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:34.928231 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84203-a30d-4b58-95a6-cb0d40e38ccd None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:34.928586 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84203-a30d-4b58-95a6-cb0d40e38ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:34.961994 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:35.006631 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-83c84203-a30d-4b58-95a6-cb0d40e38ccd None None] MFA Rules not processed for user `0a76012127184a19bd9ccab445d24d75`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:35.007872 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-83c84203-a30d-4b58-95a6-cb0d40e38ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:35.085137 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 504/1004] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Oct 17 06:13:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:35.094856 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c5f26-7308-4c4c-bf8f-40484b10f6d8 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:35.096571 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9f4c5f26-7308-4c4c-bf8f-40484b10f6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:35.207595 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c5f26-7308-4c4c-bf8f-40484b10f6d8 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:35.280933 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:35.281539 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:35.281820 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:35.282093 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:35.283749 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:35.300227 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:35.396742 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:35.398226 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c3500-2b56-4760-b831-acd10fafddc8 service neutron] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:35.416260 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 501/1005] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:37.877020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a10f-705f-409b-9259-1055c171cd26 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:37.877563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a10f-705f-409b-9259-1055c171cd26 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:37.877563 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a10f-705f-409b-9259-1055c171cd26 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:37.879125 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 505/1006] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 06:13:37.884577 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c838e-c44c-420b-b230-d392b126f80f None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:37.885052 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c838e-c44c-420b-b230-d392b126f80f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:37.885154 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c838e-c44c-420b-b230-d392b126f80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:37.950992 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-a63c838e-c44c-420b-b230-d392b126f80f None None] MFA Rules not processed for user `c4cc297532144f6f8f4534b8faf80b42`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:37.952224 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-a63c838e-c44c-420b-b230-d392b126f80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:38.017283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:38.048474 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 502/1007] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:38.338808 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a10f-705f-409b-9259-1055c171cd26 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:38.340409 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-5ee1a10f-705f-409b-9259-1055c171cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:38.444736 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a10f-705f-409b-9259-1055c171cd26 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:38.467723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:38.521758 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:38.522378 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:38.522661 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:38.522933 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:38.525338 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:38.636144 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:38.637619 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c70341-b7c0-47c3-8b82-19029e81f822 service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:38.656377 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 506/1008] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:41.400487 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678237-0c1a-4870-b3a8-57f2755d9a67 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:41.401029 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678237-0c1a-4870-b3a8-57f2755d9a67 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:41.401497 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678237-0c1a-4870-b3a8-57f2755d9a67 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:41.404219 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 503/1009] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:41.409583 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b048f-cc18-49f3-8f14-4fa47ca8e6ea None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:41.409999 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b048f-cc18-49f3-8f14-4fa47ca8e6ea None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:41.410146 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b048f-cc18-49f3-8f14-4fa47ca8e6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:41.475677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:41.478689 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-5b6b048f-cc18-49f3-8f14-4fa47ca8e6ea None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:41.479877 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-5b6b048f-cc18-49f3-8f14-4fa47ca8e6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:41.577119 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 507/1010] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:41.860187 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678237-0c1a-4870-b3a8-57f2755d9a67 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:41.861698 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-72678237-0c1a-4870-b3a8-57f2755d9a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:41.967177 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678237-0c1a-4870-b3a8-57f2755d9a67 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:42.042292 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:42.042901 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:42.043181 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:42.043491 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:42.045077 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:42.071624 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:42.153390 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:42.154750 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb558f4-d2bc-441a-8333-417ead96635f service neutron] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:42.173648 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 504/1011] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:44.400681 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352b6b-f3ec-4a09-ad7a-dbfd9208a68e None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:44.401354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352b6b-f3ec-4a09-ad7a-dbfd9208a68e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:44.401354 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352b6b-f3ec-4a09-ad7a-dbfd9208a68e None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:44.402569 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 508/1012] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:44.408173 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25807ea-7760-4b97-977f-9b83ad5e7e99 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:44.408665 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25807ea-7760-4b97-977f-9b83ad5e7e99 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:44.408768 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25807ea-7760-4b97-977f-9b83ad5e7e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:44.479354 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-c25807ea-7760-4b97-977f-9b83ad5e7e99 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:44.480586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-c25807ea-7760-4b97-977f-9b83ad5e7e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:44.558103 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:44.582373 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 505/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:44.877484 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352b6b-f3ec-4a09-ad7a-dbfd9208a68e None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:44.879042 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-de352b6b-f3ec-4a09-ad7a-dbfd9208a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:44.986269 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352b6b-f3ec-4a09-ad7a-dbfd9208a68e None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:45.014904 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:45.060723 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:45.061324 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:45.061599 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:45.061873 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:45.063504 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:45.173574 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:45.174931 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd177f-8458-4533-9c46-a6c4dbf6ecdc service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:45.193217 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 509/1014] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:48.017347 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f14d1-1195-4659-ab5f-4dacd0b5e4a5 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:48.017884 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f14d1-1195-4659-ab5f-4dacd0b5e4a5 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:48.017884 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f14d1-1195-4659-ab5f-4dacd0b5e4a5 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:48.019235 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 506/1015] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:48.024537 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70100a60-7467-4e62-ab94-6ab63efbcf4b None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:48.024988 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70100a60-7467-4e62-ab94-6ab63efbcf4b None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:48.025122 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70100a60-7467-4e62-ab94-6ab63efbcf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:48.094639 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-70100a60-7467-4e62-ab94-6ab63efbcf4b None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:48.095944 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-70100a60-7467-4e62-ab94-6ab63efbcf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:48.105087 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:48.195279 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 510/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:48.478129 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f14d1-1195-4659-ab5f-4dacd0b5e4a5 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:48.479771 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-100f14d1-1195-4659-ab5f-4dacd0b5e4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:48.585154 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f14d1-1195-4659-ab5f-4dacd0b5e4a5 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:48.661547 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:48.662163 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:48.662447 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:48.662719 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:48.664921 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:48.702653 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:48.782555 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:48.784065 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48bf1d-2949-4e26-8411-f6a796689411 service neutron] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:48.803080 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 507/1017] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:53.253428 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69e341-f4d9-495b-898d-7de7150c8cd7 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:53.253987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69e341-f4d9-495b-898d-7de7150c8cd7 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:53.253987 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69e341-f4d9-495b-898d-7de7150c8cd7 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:53.255705 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 511/1018] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:53.261269 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078947-ff0a-4f39-9b7c-8594f6dcbbb3 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:53.261673 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078947-ff0a-4f39-9b7c-8594f6dcbbb3 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:53.261853 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3078947-ff0a-4f39-9b7c-8594f6dcbbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:53.333534 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-e3078947-ff0a-4f39-9b7c-8594f6dcbbb3 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:53.334743 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-e3078947-ff0a-4f39-9b7c-8594f6dcbbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:53.425348 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:53.436762 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 508/1019] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:53.720817 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69e341-f4d9-495b-898d-7de7150c8cd7 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:53.722432 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-bb69e341-f4d9-495b-898d-7de7150c8cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:53.831542 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69e341-f4d9-495b-898d-7de7150c8cd7 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:53.872020 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:53.908867 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:53.909472 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:53.909752 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:53.910027 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:53.911632 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:54.028138 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:54.029538 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb00e19-5c1d-4c46-a83d-c55c64dc6090 service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:54.048596 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 512/1020] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:13:57.394034 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f510e-e93c-466d-bb8a-66602e6f1b03 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:57.394649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f510e-e93c-466d-bb8a-66602e6f1b03 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:57.394649 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f510e-e93c-466d-bb8a-66602e6f1b03 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:57.396157 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 509/1021] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:13:57.401420 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d6f44-a8e3-419d-b233-ef467be31a71 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:57.401820 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d6f44-a8e3-419d-b233-ef467be31a71 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:57.402010 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d6f44-a8e3-419d-b233-ef467be31a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:57.482507 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-992d6f44-a8e3-419d-b233-ef467be31a71 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:13:57.483863 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-992d6f44-a8e3-419d-b233-ef467be31a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:57.506900 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:57.594235 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 513/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:13:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:13:57.877600 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f510e-e93c-466d-bb8a-66602e6f1b03 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:13:57.879171 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-9b5f510e-e93c-466d-bb8a-66602e6f1b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:57.987440 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f510e-e93c-466d-bb8a-66602e6f1b03 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:13:58.094204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:13:58.094204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:13:58.094486 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:13:58.094572 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:13:58.096252 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:13:58.143804 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:13:58.214856 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:13:58.216339 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295030-2ee7-4d1c-a033-22e8822e44da service neutron] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:13:58.238916 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 510/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:13:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:14:02.798435 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b098d-0d33-4376-ab7f-ea6e634f5506 None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:02.799012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b098d-0d33-4376-ab7f-ea6e634f5506 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:02.799012 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b098d-0d33-4376-ab7f-ea6e634f5506 None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:02.800572 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 514/1024] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:02.806306 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6def1-b36a-42ce-8c51-ccb0769aa854 None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:02.806811 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6def1-b36a-42ce-8c51-ccb0769aa854 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:02.806926 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6def1-b36a-42ce-8c51-ccb0769aa854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:02.888105 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-dfe6def1-b36a-42ce-8c51-ccb0769aa854 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:02.889329 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-dfe6def1-b36a-42ce-8c51-ccb0769aa854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:02.999119 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:03.001876 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 511/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:03.286581 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b098d-0d33-4376-ab7f-ea6e634f5506 None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:14:03.288835 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-df8b098d-0d33-4376-ab7f-ea6e634f5506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:03.406289 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b098d-0d33-4376-ab7f-ea6e634f5506 None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:14:03.457265 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:03.483917 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a76012127184a19bd9ccab445d24d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:14:03.484525 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:03.484800 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:03.485067 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:03.486649 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:03.596359 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:14:03.597784 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abdb90-b111-4c62-b4e7-61d5fedbf83d service neutron] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:14:03.616457 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 515/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:14:09.419468 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0cbab-67ba-400c-b98f-3f6c03e5a8fd None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:09.419993 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0cbab-67ba-400c-b98f-3f6c03e5a8fd None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:09.420172 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0cbab-67ba-400c-b98f-3f6c03e5a8fd None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:09.421968 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 512/1027] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 06:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:09.428706 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725adffd-221a-481a-a06a-d52fe8bee9c9 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:09.441800 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725adffd-221a-481a-a06a-d52fe8bee9c9 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:09.441800 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725adffd-221a-481a-a06a-d52fe8bee9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:09.583667 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-725adffd-221a-481a-a06a-d52fe8bee9c9 None None] MFA Rules not processed for user `e588341177544d43a4bdf82cc1107db1`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:09.586666 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-725adffd-221a-481a-a06a-d52fe8bee9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:09.634838 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:09.837570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 516/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 06:14:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:15.097846 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad226-5261-4811-8171-54b3bcedc84c None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:15.098390 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad226-5261-4811-8171-54b3bcedc84c None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:15.098390 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad226-5261-4811-8171-54b3bcedc84c None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:15.106822 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 513/1029] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 06:14:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:15.112925 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42994-c21c-42f5-b389-11583b5a79bc None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:15.115023 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42994-c21c-42f5-b389-11583b5a79bc None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:15.115584 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42994-c21c-42f5-b389-11583b5a79bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:15.207692 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-c1c42994-c21c-42f5-b389-11583b5a79bc None None] MFA Rules not processed for user `e588341177544d43a4bdf82cc1107db1`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:15.208911 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-c1c42994-c21c-42f5-b389-11583b5a79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:15.296906 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:15.343674 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 517/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 06:14:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:16.399325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034757d6-1698-48c1-b519-df1191b2060a None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:16.399325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034757d6-1698-48c1-b519-df1191b2060a None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:16.399325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034757d6-1698-48c1-b519-df1191b2060a None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:16.400713 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 514/1031] 10.222.0.13 () {54 vars in 921 bytes} [Mon Oct 17 06:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:16.405843 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450336c-1166-4db1-9c64-450794267ff9 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:16.406360 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450336c-1166-4db1-9c64-450794267ff9 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:16.406552 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450336c-1166-4db1-9c64-450794267ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:16.491181 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-4450336c-1166-4db1-9c64-450794267ff9 None None] MFA Rules not processed for user `e588341177544d43a4bdf82cc1107db1`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:16.492339 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-4450336c-1166-4db1-9c64-450794267ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:16.612070 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 518/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Oct 17 06:14:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:20.253099 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f08c-10e3-4921-b407-0c0c2a856d83 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:20.255170 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f08c-10e3-4921-b407-0c0c2a856d83 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:20.255512 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f08c-10e3-4921-b407-0c0c2a856d83 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:20.257019 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 515/1033] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:14:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:20.263173 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906fb11-abe4-4c0a-93d9-de846bc31710 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:20.263601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906fb11-abe4-4c0a-93d9-de846bc31710 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:20.263601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906fb11-abe4-4c0a-93d9-de846bc31710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:20.312074 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:20.364133 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-7906fb11-abe4-4c0a-93d9-de846bc31710 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:20.368826 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-7906fb11-abe4-4c0a-93d9-de846bc31710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:20.540271 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 519/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:20.859041 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1f40e-5c73-44aa-979e-6c0469234e0f None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:20.859639 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1f40e-5c73-44aa-979e-6c0469234e0f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:20.860280 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1f40e-5c73-44aa-979e-6c0469234e0f None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:20.861791 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 516/1035] 10.222.0.13 () {54 vars in 961 bytes} [Mon Oct 17 06:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:20.873200 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763455da-40a6-49ab-9701-c404e7a27317 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:20.873515 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763455da-40a6-49ab-9701-c404e7a27317 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:20.873791 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763455da-40a6-49ab-9701-c404e7a27317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:20.966136 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-763455da-40a6-49ab-9701-c404e7a27317 None None] MFA Rules not processed for user `9dcc0e6649de439eb130a3b5f5bc1b91`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:20.967361 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-763455da-40a6-49ab-9701-c404e7a27317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:21.023831 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:21.096990 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 520/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Oct 17 06:14:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:21.105770 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1f40e-5c73-44aa-979e-6c0469234e0f None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:14:21.107356 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-6ac1f40e-5c73-44aa-979e-6c0469234e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:21.300665 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1f40e-5c73-44aa-979e-6c0469234e0f None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:14:21.390535 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:21.441469 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcc0e6649de439eb130a3b5f5bc1b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:14:21.442550 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:21.443343 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:21.444432 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:21.446623 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:21.544749 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:14:21.546343 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d85eb-62c3-416e-97b4-03b975dfd5f1 service nova] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:14:21.565886 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 517/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:14:23.991077 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7487aa4-ec5f-474d-b825-efe7c49a939e None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:23.991552 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7487aa4-ec5f-474d-b825-efe7c49a939e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:23.991635 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7487aa4-ec5f-474d-b825-efe7c49a939e None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:23.993280 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 521/1038] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:24.003583 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebeb2-32c8-4a5d-a241-7658f22f3f2f None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:24.005766 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebeb2-32c8-4a5d-a241-7658f22f3f2f None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:24.006283 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ebeb2-32c8-4a5d-a241-7658f22f3f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:24.091658 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-311ebeb2-32c8-4a5d-a241-7658f22f3f2f None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:24.092862 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-311ebeb2-32c8-4a5d-a241-7658f22f3f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:24.143228 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:24.245772 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 518/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:24.539549 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7487aa4-ec5f-474d-b825-efe7c49a939e None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:14:24.542849 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-e7487aa4-ec5f-474d-b825-efe7c49a939e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:24.822595 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7487aa4-ec5f-474d-b825-efe7c49a939e None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:14:24.836212 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:24.995601 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcc0e6649de439eb130a3b5f5bc1b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:14:24.997308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:24.997772 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:24.998197 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:25.000029 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:25.127341 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:14:25.129506 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a229881-3382-4ecf-a566-015fb03fabb9 service nova] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:14:25.185255 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 522/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:14:27.947678 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31110b7-5aa4-419f-94bc-c74542163c18 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:27.948248 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31110b7-5aa4-419f-94bc-c74542163c18 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:27.948248 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31110b7-5aa4-419f-94bc-c74542163c18 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:27.949587 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 519/1041] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:27.955179 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f850-b2e4-4731-b061-28ef270b8836 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:27.955677 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f850-b2e4-4731-b061-28ef270b8836 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:27.955991 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f850-b2e4-4731-b061-28ef270b8836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:27.974391 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:28.033430 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-02e9f850-b2e4-4731-b061-28ef270b8836 None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:28.034936 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-02e9f850-b2e4-4731-b061-28ef270b8836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:28.149326 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 523/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:28.457204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804812-b54e-41bc-bf98-c884febde479 None None] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:28.457204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804812-b54e-41bc-bf98-c884febde479 None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:28.457204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804812-b54e-41bc-bf98-c884febde479 None None] PATH_INFO: `/` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:28.457204 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 520/1043] 10.222.0.13 () {54 vars in 961 bytes} [Mon Oct 17 06:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:28.478113 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed57c1d-58ab-4233-996f-131c98a59614 None None] REQUEST_METHOD: `POST` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:28.478570 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed57c1d-58ab-4233-996f-131c98a59614 None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:28.478956 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed57c1d-58ab-4233-996f-131c98a59614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:28.578649 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.auth.core [None req-0ed57c1d-58ab-4233-996f-131c98a59614 None None] MFA Rules not processed for user `9dcc0e6649de439eb130a3b5f5bc1b91`. Rule list: `[]` (Enabled: `True`). {{(pid=218501) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:28.580308 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-0ed57c1d-58ab-4233-996f-131c98a59614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:28.629323 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:28.776979 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 524/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Oct 17 06:14:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:28.785137 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0804812-b54e-41bc-bf98-c884febde479 None None] Authenticating user token {{(pid=218502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:14:28.786962 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-f0804812-b54e-41bc-bf98-c884febde479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:28.976337 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0804812-b54e-41bc-bf98-c884febde479 None None] Validating token access rules against request {{(pid=218502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:14:29.008418 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:29.116429 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcc0e6649de439eb130a3b5f5bc1b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:14:29.117058 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] REQUEST_METHOD: `GET` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:29.117335 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:29.117606 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:29.119386 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:29.229549 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:14:29.231346 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab472e90-f27e-4b52-a452-b10c0b9ebc1c service nova] RBAC: Authorization granted {{(pid=218502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:14:29.255382 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 521/1045] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:14:29.380628 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d05e08-5959-40b3-8b18-965a794fc11b None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:29.381186 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d05e08-5959-40b3-8b18-965a794fc11b None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:29.381186 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d05e08-5959-40b3-8b18-965a794fc11b None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:29.382814 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 525/1046] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:29.388023 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba38e8a-6d0f-4550-9213-7a4d8e07e6bd None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:29.388325 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba38e8a-6d0f-4550-9213-7a4d8e07e6bd None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:29.388586 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba38e8a-6d0f-4550-9213-7a4d8e07e6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:29.424355 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:29.475481 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-2ba38e8a-6d0f-4550-9213-7a4d8e07e6bd None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:29.476720 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-2ba38e8a-6d0f-4550-9213-7a4d8e07e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:29.601951 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 522/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:31.766865 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd32aa-76ca-4d54-836b-151441fdc91e None None] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:31.767534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd32aa-76ca-4d54-836b-151441fdc91e None None] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:31.767534 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd32aa-76ca-4d54-836b-151441fdc91e None None] PATH_INFO: `/` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:31.768745 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 526/1048] 10.222.0.13 () {54 vars in 927 bytes} [Mon Oct 17 06:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 06:14:31.774412 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd899b-c25c-4763-9c8b-5791136997fe None None] REQUEST_METHOD: `POST` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:31.774874 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd899b-c25c-4763-9c8b-5791136997fe None None] SCRIPT_NAME: `/identity` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:31.774969 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd899b-c25c-4763-9c8b-5791136997fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=218502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:31.845870 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.auth.core [None req-f7cd899b-c25c-4763-9c8b-5791136997fe None None] MFA Rules not processed for user `bf714812a1a846729a9dd3749da0e163`. Rule list: `[]` (Enabled: `True`). {{(pid=218502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 06:14:31.847083 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG keystone.common.fernet_utils [None req-f7cd899b-c25c-4763-9c8b-5791136997fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:31.905379 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=100 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:31.950738 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: [pid: 218502|app: 0|req: 523/1049] 10.222.0.13 () {58 vars in 1024 bytes} [Mon Oct 17 06:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 06:14:32.242701 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd32aa-76ca-4d54-836b-151441fdc91e None None] Authenticating user token {{(pid=218501) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 06:14:32.244412 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-c3cd32aa-76ca-4d54-836b-151441fdc91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:32.394958 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:32.401260 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd32aa-76ca-4d54-836b-151441fdc91e None None] Validating token access rules against request {{(pid=218501) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 06:14:32.532522 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcc0e6649de439eb130a3b5f5bc1b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6727b897172440d8bdc7c2d81ea3d9da', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=218501) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 06:14:32.533307 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] REQUEST_METHOD: `GET` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 06:14:32.533618 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] SCRIPT_NAME: `/identity` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 06:14:32.533916 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=218501) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 06:14:32.535731 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.fernet_utils [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=218501) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 06:14:32.640006 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 06:14:32.641932 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d20eb-8837-4a67-854d-0cf8c778e134 service nova] RBAC: Authorization granted {{(pid=218501) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 06:14:32.657871 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: DEBUG dbcounter [-] [218501] Writing DB stats keystone:SELECT=100 {{(pid=218501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:32.660895 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: [pid: 218501|app: 0|req: 527/1050] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Oct 17 06:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 06:14:41.948912 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: DEBUG dbcounter [-] [218502] Writing DB stats keystone:SELECT=19 {{(pid=218502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 06:14:52.909075 dv-cinder-861566-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... Oct 17 06:14:52.909988 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: Mon Oct 17 06:14:52 2022 - graceful shutdown triggered... Oct 17 06:14:52.910440 dv-cinder-861566-1-iscsi devstack@keystone.service[218501]: Gracefully killing worker 1 (pid: 218501)... Oct 17 06:14:52.911413 dv-cinder-861566-1-iscsi devstack@keystone.service[218502]: Gracefully killing worker 2 (pid: 218502)... Oct 17 06:14:53.909815 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: worker 1 buried after 1 seconds Oct 17 06:14:53.909815 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: worker 2 buried after 1 seconds Oct 17 06:14:53.909815 dv-cinder-861566-1-iscsi devstack@keystone.service[218489]: goodbye to uWSGI. Oct 17 06:14:53.911283 dv-cinder-861566-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. Oct 17 06:14:53.911886 dv-cinder-861566-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.